REANA-Server

image image image image image image

REANA-Server is a component of the REANA reusable and reproducible research data analysis platform. It implements the API Server that takes and performs REST API calls issued by REANA clients.

Features

  • offers rich REST API services for REANA clients

  • transmits REST API requests to appropriate REANA cloud components

  • REST API to run research analysis workflows on compute clouds

  • REST API to list submitted workflows and enquire about their statuses

  • REST API to manage analysis files

  • REST API to download results of finished analysis workflows

  • REST API to find the differences between two workflows (git like output)

Usage

The detailed information on how to install and use REANA can be found in docs.reana.io.

Authentication and revocation lifecycle

Upgrading existing user identities

Accounts created before OIDC/JWT authentication have no linked identity (idp_issuer/idp_subject are unset) and cannot sign in through the new flow until linked to an identity from the configured issuer. On upgrade each such user’s first login fails with a clear “an account already exists” ProvisioningError rather than a confusing crash, but they remain unable to sign in until an administrator resolves it — this affects every pre-existing account on any installation with real users, not just an edge case. Fresh installs are unaffected.

Two ways to resolve it, and they can be combined:

  • Link accounts by hand, one at a time, as they need access again:

    $ flask reana-admin create-admin-user \
      --email jane.doe@example.org \
      --idp-issuer https://issuer.example.org \
      --idp-subject oidc-subject

    (works for any existing user, not only admins — see flask reana-admin --help).

  • Enable automatic email-based linking, so a user whose verified IdP email matches their existing REANA account’s email is linked on first login with no administrator action:

    REANA_AUTH_EMAIL_LINKING_ENABLED=true
REANA_AUTH_EMAIL_LINKING_ISSUER_ALLOWLIST=https://issuer.example.org
REANA_AUTH_EMAIL_LINKING_DOMAIN_ALLOWLIST=example.org

    Disabled by default. This trusts the issuer’s email_verified claim as proof of ownership of a pre-existing REANA account’s email address, so only enable it for issuers you control or otherwise trust to verify email ownership correctly. Both allow-lists are optional and independently applied; an empty allow-list skips that particular check rather than rejecting every issuer/domain.

    Some institutional issuers (e.g. CERN Keycloak) never emit the standard OIDC email_verified claim at all, even though their email is verified out-of-band by the issuer itself. REANA_AUTH_EMAIL_LINKING_ASSUME_VERIFIED_ISSUERS is a comma-separated list of issuers for which the administrator explicitly attests that their asserted email can be trusted without that claim; it does not weaken the check for any other issuer.

REANA Server validates API access tokens statelessly. Issuer, audience, signature, expiry, and the configured REANA role are checked on every protected request. Consequently, role removal or account disablement takes effect for direct API access when the currently issued access token expires; deployments that need a shorter revocation window must configure that lifetime at the identity provider.

During a transient issuer outage, REANA can continue validating known signing keys from its last successfully fetched JWKS for REANA_AUTH_JWKS_STALE_GRACE seconds beyond the normal REANA_AUTH_JWKS_TTL (one hour beyond a ten-minute TTL by default). Once that bounded grace period ends, authentication fails with an issuer-availability error until current keys can be fetched. Set the stale grace to 0 to disable the outage fallback after the normal TTL.

Browser BFF sessions follow the same access-token rule. Refresh tokens are kept only in Redis, bound to issuer, subject, and web client, and are removed after REANA_AUTH_SESSION_TTL even if the issuer would keep them longer. Refresh is accepted only when the issuer returns an access token for the same identity; the refreshed token’s current REANA role is checked before serving the request. Operators should configure the issuer to reject refresh for disabled accounts.

An already running interactive notebook uses an independent per-session secret. Identity-provider revocation does not automatically terminate that workload. Immediate revocation therefore requires closing the interactive session through REANA, or immediately for every session a user has open:

$ flask reana-admin interactive-session-cleanup --email jane.doe@example.org

(--id also accepted). This closes sessions right away regardless of activity, unlike --days, which only closes sessions inactive for longer than the given number of days. The configured inactivity cleanup (--days) is the automatic bound; a value of forever deliberately provides no automatic revocation bound and is not suitable where immediate account revocation is required – use --email/--id for that case instead.

Legacy opaque REANA tokens are not accepted as GitLab webhook credentials. On upgrade from an opaque-token release, users must delete and recreate existing GitLab webhooks through REANA once. The new hook receives a separately generated encrypted per-user secret; no login or API credential is reused.

The secret is a delegated, time-limited capability rather than an OIDC token. Its authorization expires after REANA_GITLAB_WEBHOOK_SECRET_MAX_LIFETIME seconds (30 days by default), at which point webhook deliveries are rejected until the user signs in to REANA with their current identity-provider entitlement and explicitly renews the authorization from their profile. Renewal changes only the expiry timestamp; it deliberately preserves the secret already installed in all of the user’s GitLab projects. A longer configured lifetime reduces how often users must renew, but also increases the maximum time for which a user who loses their REANA role can continue launching workflows through GitLab. Secrets created before expiry enforcement have no expiry and fail closed until first renewal.

Operators upgrading an active installation should expect every pre-existing GitLab webhook to fail closed immediately on upgrade: the new validator accepts only the per-user secret with a future expiry, and no existing hook carries one until its user renews (or recreates it). When deliveries are rejected for long enough, GitLab may auto-disable the hook — temporarily after a few consecutive failures and permanently after many. Renewing the REANA authorization restores REANA’s acceptance, but a hook GitLab has permanently disabled additionally needs a successful test delivery from GitLab before it resumes. REANA cannot detect or repair that disablement itself, so renewing an already-expired authorization (PUT /api/gitlab/webhook-token) returns a message field warning of it, as a pointer back to this manual recovery step. Plan upgrades of busy installations accordingly, and tell users to renew promptly so their projects do not cross GitLab’s auto-disable thresholds during an unattended expiry window.

Administrators do not have to wait out an expiry window, but the correct procedure depends on the incident.

Offboarding or a suspected account compromise. Remove the user’s identity-provider entitlement (their reana:user role) first — a live access token keeps working until it expires regardless of anything below, since REANA validates bearer tokens statelessly — then revoke everything REANA itself can revoke immediately, in one command:

$ reana-admin revoke-identity --email user@example.org

This closes the user’s open interactive sessions, revokes their GitLab webhook authorization, and deletes their browser (BFF) sessions, attempting all three independently so a failure in one does not block the others. Equivalent to running interactive-session-cleanup --email, gitlab-webhook-revoke --email, and a BFF-session revocation separately. Pass --delete-secret to also permanently invalidate the GitLab webhook secret already installed in the user’s projects (see below), and --dry-run to preview without changing anything. The ordering above still matters: while the account still holds the required role it can sign in and undo some of this (e.g. renew the webhook authorization), so revoking before the entitlement is removed leaves a window in which the user restores their own access.

The narrower commands below remain available for standalone use — e.g. only a leaked-secret response, without touching interactive or browser sessions.

A leaked webhook secret. When the secret value itself must be treated as compromised, revoke and delete it directly:

$ reana-admin gitlab-webhook-revoke --email user@example.org --delete-secret

--delete-secret permanently invalidates every hook already installed in GitLab and forces the user to re-enable each project; it is the operation that invalidates a captured secret value. It does not by itself close the offboarding window — an account that still holds the role can create a fresh secret the next time it enables a project — so for offboarding use revoke-identity (with the identity-provider entitlement removed first) as above, or run this command with --delete-secret after doing so.

Without --delete-secret, gitlab-webhook-revoke only de-authorizes the secret — clearing its expiry — while keeping it, so nothing needs to be reconfigured in GitLab.

--dry-run reports the effect of either form without applying it.

CLI API

flask reana-admin

REANA administration commands.

Usage

flask reana-admin [OPTIONS] COMMAND [ARGS]...

check-workflows

Check consistency of selected workflow run statuses between database, message queue and Kubernetes.

Usage

flask reana-admin check-workflows [OPTIONS]

Options

--date-start <date_start>

Default value is 24 hours ago.

--date-end <date_end>

Default value is now.

-a, --all

Show all workflows/sessions/workspaces, even if in-sync.

create-admin-user

Create the default administrator user.

Credentials are owned by the OIDC issuer (e.g. the bundled Keycloak); pass both identity options to create or update a pre-linked REANA row. Omit both only when an external issuer’s immutable subject is not yet available, then rerun this command with both options before login.

Usage

flask reana-admin create-admin-user [OPTIONS]

Options

-e, --email <email>

Required The email of the admin user.

-i, --id <id_>
--idp-issuer <idp_issuer>

OIDC issuer URL to link explicitly to the administrator.

--idp-subject <idp_subject>

OIDC subject to link explicitly to the administrator.

gitlab-webhook-revoke

Revoke a user’s delegated GitLab webhook authorization immediately.

Webhook secrets are delegated capabilities that expire on their own after REANA_GITLAB_WEBHOOK_SECRET_MAX_LIFETIME. This command ends one now, without waiting for that bound, so that an offboarded or compromised account stops launching workflows from GitLab straight away.

By default the secret is kept but de-authorized: the user can restore it from the REANA web interface, which re-checks their current identity provider entitlement. Use --delete-secret when the secret itself must be considered compromised.

Usage

flask reana-admin gitlab-webhook-revoke [OPTIONS]

Options

--delete-secret

Also delete the secret. Hooks already installed in GitLab stop working permanently and the user must recreate them through REANA.

--dry-run

Report what would be revoked without committing the change.

-e, --email <email>

The email of the user.

--id <id_>

The id of the user.

interactive-session-cleanup

Close inactive interactive sessions, or one user’s immediately.

Without --email/--id, closes sessions inactive for more than --days. With --email/--id, ignores --days and closes every one of that user’s active sessions right away – this is REANA’s way to make interactive-session revocation immediate (rather than bounded only by the configured inactivity cleanup) when offboarding a user or responding to a suspected account compromise, matching gitlab-webhook-revoke’s role in the webhook revocation lifecycle.

Usage

flask reana-admin interactive-session-cleanup [OPTIONS]

Options

-d, --days <days>

Close interactive sessions that are inactive for more than the specified number of days.

--dry-run

Show which interactive sessions would be closed, without closing them. [default=False]

-e, --email <email>

The email of the user.

--id <id_>

The id of the user.

queue-consume

Start consuming specified queue and remove selected messages.

By default, you will need to specify either “-k” or “-i” options otherwise the command will return an error.

If -k option is specified, messages that have property values specified in -v will be deleted.

If -i option is specified, for every message, user will be asked what to do.

If -k and -i are specified together, for every message that matches property values in -v, user will be asked whether to delete it or not.

Usage

flask reana-admin queue-consume [OPTIONS]

Options

-q, --queue-name <queue_name>

Required Name of the queue that will be consumed, e.g workflow-submission

-k, --key <key>

Key of the property that will be used to filter the messages in the queue, e.g workflow_name_or_id

-v, --values-to-delete <values_to_delete>

List of property values used to filter messages that will be removed from the queue, e.g UUID of a workflow

-i, --interactive

Manually decide which messages to remove from the queue.

quota-resources

List available quota resources.

Usage

flask reana-admin quota-resources [OPTIONS]

quota-set

Set quota limits to the given users per resource.

Usage

flask reana-admin quota-set [OPTIONS]

Options

-e, --email <emails>

Required The emails of the users. E.g. –email johndoe@example.org –email janedoe@example.org

-r, --resource <resource_type>

Specify quota resource. e.g. cpu, disk.

-n, --resource-name <resource_name>

Name of resource.

-l, --limit <limit>

Required New limit in canonical unit.

quota-set-default-limits

Set default quota limits for users who do not have any custom limits

defined.

Note that any previously set user limits, either via old defaults or via custom settings, will be kept during the upgrade, and won’t be automatically updated to match the new default limit value.

Usage

flask reana-admin quota-set-default-limits [OPTIONS]

quota-set-period

Set periodic quota fields for a given user.

Usage

flask reana-admin quota-set-period [OPTIONS]

Options

--id <user_id>

The id of the user.

-e, --email <email>

The email of the user.

--resource <resource_type>

Required Specify quota resource. Only cpu is currently supported.

Options:

cpu

--quota-period-months <quota_period_months>

Length of quota period in months.

--quota-period-start-at <quota_period_start_at>

Current active quota period start datetime.

quota-usage

List quota usage of users.

Usage

flask reana-admin quota-usage [OPTIONS]

Options

--id <id>

The id of the user.

-e, --email <email>

The email of the user.

--json

Get output in JSON format.

-h, --human-readable

Show quota usage values in human readable format.

retention-rules-apply

Apply pending retentions rules.

Usage

flask reana-admin retention-rules-apply [OPTIONS]

Options

--dry-run

Show the pending retention rules without applying them. [default=False]

--force-date <force_date>

Force desired date and time when deciding which rules to apply.

--yes-i-am-sure

Do not ask for confirmation when doing potentially dangerous operations.

-e, --email <email>

The email of the user.

--id <id_>

The id of the user.

-w, --workflow <workflow_uuid>

The id of the workflow.

retention-rules-extend

Extend active retentions rules.

Usage

flask reana-admin retention-rules-extend [OPTIONS]

Options

-w, --workflow <workflow_uuid>

Required The id of the workflow.

-d, --days <days>

Required Number of days to extend the rules.

revoke-identity

Revoke every REANA-side session and secret for one identity at once.

Closes the user’s open interactive sessions immediately, revokes their GitLab webhook authorization, and deletes their browser (BFF) sessions – everything REANA itself can revoke without delay, in one command instead of three. Equivalent to running interactive-session-cleanup --email, gitlab-webhook-revoke --email, and a BFF-session revocation separately; see those commands for narrower standalone use (e.g. only closing sessions, or only a leaked-secret response).

The three subsystems are independent (Kubernetes, the database, and Redis respectively) and are each attempted even if another one fails – an outage in one must not leave the others un-revoked. The GitLab webhook and BFF-session steps run first since neither depends on Kubernetes; the exit code is non-zero if any subsystem failed, after every subsystem has been attempted.

This does NOT remove the user’s identity-provider role/entitlement, and cannot revoke a JWT access token already issued: REANA validates bearer tokens statelessly, so a live one keeps working until it expires regardless of anything this command does. For offboarding or a suspected compromise, remove the identity-provider role FIRST, then run this – the same ordering gitlab-webhook-revoke’s documentation already requires, now covering every REANA-side credential at once.

Usage

flask reana-admin revoke-identity [OPTIONS]

Options

--delete-secret

Also delete the GitLab webhook secret. Hooks already installed in GitLab stop working permanently and the user must recreate them through REANA.

--dry-run

Report what would be revoked without changing or deleting anything.

-e, --email <email>

The email of the user.

--id <id_>

The id of the user.

status-report

Get a status report of the REANA system.

Usage

flask reana-admin status-report [OPTIONS]

Options

--type <types>

Type of information to be displayed?

Options:

interactive-sessions | workflows | users | system | storage | nodes | pods | jobs | quota-usage | all

-e, --email <email>

Send the status by email to the configured receiver.

user-create

Create a new user.

Usage

flask reana-admin user-create [OPTIONS]

Options

-e, --email <email>

Required The email of the user.

user-export

Export all users in current REANA cluster.

Usage

flask reana-admin user-export [OPTIONS]

user-import

Import users from file.

Usage

flask reana-admin user-import [OPTIONS]

Options

-f, --file <file_>

A CSV file containing a list of REANA users.

user-list

List users according to the search criteria.

Usage

flask reana-admin user-list [OPTIONS]

Options

--id <id>

The id of the user.

-e, --email <email>

The email of the user.

--json

Get output in JSON format.

REST API

The REANA Server offers a REST API for management workloads (workflows, jobs, tasks, etc.) running on REANA Cloud. Detailed REST API documentation can be found here.

Reana-Server Ping-functionality Flask-Blueprint.

reana_server.rest.ping.health()[source]

Report health of REANA Server’s auth-adjacent dependencies.

reana_server.rest.ping.ping()[source]

Ping the server (healthcheck)

Reana-Server User Endpoints.

reana_server.rest.users.get_users_shared_with_you(user)[source]

Gets users that shared workflow(s) with the authenticated user.

reana_server.rest.users.get_users_you_shared_with(user)[source]

Gets users that the authenticated user shared workflow(s) with.

reana_server.rest.users.get_you(user)[source]

Gets information about authenticated user.

Reana-Server workflow-functionality Flask-Blueprint.

reana_server.rest.workflows.close_interactive_session(workflow_id_or_name, user)[source]

Close an interactive workflow session.

reana_server.rest.workflows.create_workflow(user)[source]

Creates a new workflow based on a REANA specification file.

reana_server.rest.workflows.delete_file(workflow_id_or_name, file_name, user)[source]

Delete the specified file.

reana_server.rest.workflows.download_file(workflow_id_or_name, file_name, user)[source]

Returns the requested file.

reana_server.rest.workflows.get_files(workflow_id_or_name, user, **kwargs)[source]

Returns the workspace file list.

reana_server.rest.workflows.get_interactive_session_secret(workflow_id_or_name, user)[source]

Get the access secret of the open interactive session.

reana_server.rest.workflows.get_workflow_diff(workflow_id_or_name_a, workflow_id_or_name_b, user)[source]

Get diff between two workflows.

reana_server.rest.workflows.get_workflow_disk_usage(workflow_id_or_name, user)[source]

Get disk usage of a workflow.

reana_server.rest.workflows.get_workflow_logs(workflow_id_or_name, user, **kwargs)[source]

Get workflow logs of a workflow.

reana_server.rest.workflows.get_workflow_parameters(workflow_id_or_name, user)[source]

Get parameters of a workflow.

reana_server.rest.workflows.get_workflow_retention_rules(workflow_id_or_name, user)[source]

Get the retention rules of a workflow.

reana_server.rest.workflows.get_workflow_share_status(workflow_id_or_name, user)[source]

Get the share status of a workflow.

reana_server.rest.workflows.get_workflow_specification(workflow_id_or_name, user)[source]

Get the specification used for this workflow run.

reana_server.rest.workflows.get_workflow_status(workflow_id_or_name, user)[source]

Get status of a workflow.

reana_server.rest.workflows.get_workflows(user, **kwargs)[source]

Returns list of all current workflows in REANA.

reana_server.rest.workflows.move_files(workflow_id_or_name, user)[source]

Move files within workspace.

reana_server.rest.workflows.open_interactive_session(workflow_id_or_name, interactive_session_type, user)[source]

Start an interactive session inside the workflow workspace.

reana_server.rest.workflows.prune_workspace(workflow_id_or_name, user, include_inputs=False, include_outputs=False)[source]

Prune the workspace’s files.

reana_server.rest.workflows.restart_workflow(workflow_id_or_name, user)[source]

Restart a workflow with a replacement specification.

reana_server.rest.workflows.set_workflow_status(workflow_id_or_name, user, status, **parameters)[source]

Set status of a workflow.

reana_server.rest.workflows.share_workflow(workflow_id_or_name, user, **kwargs)[source]

Share a workflow with another user.

reana_server.rest.workflows.start_workflow(workflow_id_or_name, user, **parameters)[source]

Start workflow.

reana_server.rest.workflows.unshare_workflow(workflow_id_or_name, user, user_email_to_unshare_with)[source]

Unshare a workflow with another user.

reana_server.rest.workflows.upload_file(workflow_id_or_name, user)[source]

Adds a file to the workspace.

reana_server.rest.workflows.validate_workflow_specification(user)[source]

Validate a raw workflow specification bundle.

Changelog

0.9.4 (2024-11-29)

Build

  • python: bump shared REANA packages as of 2024-11-28 (#714) (94fbf77)

Features

  • config: make ACCOUNTS_USERINFO_HEADERS customisable (#713) (8c01d51)

  • config: make APP_DEFAULT_SECURE_HEADERS customisable (#713) (1919358)

  • config: make PROXYFIX_CONFIG customisable (#713) (5b6c276)

  • config: support password-protected redis (#713) (a2aad8a)

  • ext: improve error message for db decryption error (#713) (bbab1bf)

Bug fixes

  • config: do not set DEBUG programmatically (#713) (c98cbc1)

  • config: read secret key from env (#713) (6ee6422)

  • get_workflow_specification: avoid returning null parameters (#689) (46633d6)

  • reana-admin: respect service domain when cleaning sessions (#687) (ede882d)

  • set_workflow_status: publish workflows to submission queue (#691) (6e35bd7), closes #690

  • start: validate endpoint parameters (#689) (d2d3673)

Continuous integration

  • commitlint: improve checking of merge commits (#689) (69f45fc)

0.9.3 (2024-03-04)

Build

  • deps: pin invenio-userprofiles to 1.2.4 (#665) (d6cb168)

  • docker: non-editable submodules in “latest” mode (#656) (d16fefb)

  • python: bump all required packages as of 2024-03-04 (#674) (f40b82f)

  • python: bump shared modules (#676) (47ad3ca)

  • python: bump shared REANA packages as of 2024-03-04 (#674) (aa18394)

Code refactoring

Code style

Continuous integration

  • commitlint: addition of commit message linter (#665) (2b43ecc)

  • commitlint: allow release commit style (#675) (e0299ef)

  • commitlint: check for the presence of concrete PR number (#669) (87c6145)

  • pytest: move to PostgreSQL 14.10 (#672) (e888ddd)

  • release-please: initial configuration (#665) (1d5e7c5)

  • release-please: update version in Dockerfile/OpenAPI specs (#668) (3b3dc41)

  • shellcheck: fix exit code propagation (#669) (d7eac6b)

Documentation

  • authors: complete list of contributors (#673) (71b3f38)

0.9.2 (2023-12-12)

  • Adds automated multi-platform container image building for amd64 and arm64 architectures.

  • Adds metadata labels to Dockerfile.

  • Changes workflow scheduler logging behaviour to also report the main reason behind scheduling errors to the users.

  • Fixes runtime uWSGI warning by rebuilding uWSGI with the PCRE support.

0.9.1 (2023-09-27)

  • Adds new prune_workspace endpoint to allow users to delete all the files of a workflow, specifying whether to also delete the inputs and/or the outputs.

  • Adds new interactive-session-cleanup command that can be used by REANA administrators to close interactive sessions that are inactive for more than the specified number of days.

  • Adds logic to support SSO with third-party Keycloak authentication services.

  • Adds the timestamp of when the workflow was stopped (run_stopped_at) to the workflow list and the workflow status endpoints.

  • Adds the content of the REANA_GITLAB_HOST environment variable to the list of GitLab instances from which it is possible to launch a workflow.

  • Adds progress meter to the logs of the periodic quota updater.

  • Changes CPU and disk quota calculations to improve the performance of periodic quota updater.

  • Changes the system status report to simplify and clarify the disk usage summary.

  • Changes check-workflows command to also check the presence of workspaces on the shared volume.

  • Changes check-workflows command to not show in-sync runs by default. If needed, they can be shown using the new --show-all option.

  • Changes launch endpoint to also include the warnings of the validation of the workflow specification.

  • Changes OpenAPI specification of the info endpoint to return the maximum inactivity time before automatic closure of interactive sessions.

  • Changes apispec dependency version in order to be compatible with PyYAML v6.

  • Changes reana-admin command options to require the passing of --admin-access-token argument more globally.

  • Fixes the workflow priority calculation to avoid workflows stuck in the queued status when the number of allowed concurrent workflow is set to zero.

  • Fixes GitLab integration to automatically redirect the user to the correct URL when the access request is accepted.

  • Fixes quota-set-default-limits command to propagate default quota limits to all users without custom quota limit values.

  • Fixes authentication flow to correctly deny access to past revoked tokens in case the same user has also other new active tokens.

  • Fixes email templates to show the correct kubectl commands when REANA is deployed inside a non-default namespace or with a custom component name prefix.

  • Fixes email sender for system emails to notifications.email_config.sender Helm value.

  • Fixes email receiver for token request emails to use notifications.email_config.receiver Helm value.

  • Fixes start-scheduler command to gracefully stop when being terminated.

  • Fixes container image names to be Podman-compatible.

0.9.0 (2023-01-19)

  • Adds new /api/launch endpoint that allows running workflows from remote sources.

  • Adds new get_workflow_retention_rules endpoint that allows to retrieve the workspace file retention rules of a workflow.

  • Adds queue-consume command that can be used by REANA administrators to remove specific messages from the queue.

  • Adds configuration environment variable to set an API rate limit for slow endpoints (REANA_RATELIMIT_SLOW).

  • Adds REANA specification validation utilities.

  • Adds retention-rules-apply command that can be used by REANA administrators to apply pending retention rules.

  • Adds retention-rules-extend command that can be used by REANA administrators to extend the duration of active retentions rules.

  • Adds check-workflows command that can be used by REANA administrators to check for out-of-sync workflows and interactive sessions.

  • Changes OpenAPI specification to include missing response schema elements and some other small enhancements.

  • Changes /api/info endpoint to also include the kubernetes maximum memory limit, the kubernetes default memory limit and the maximum workspace retention period.

  • Changes start_workflow endpoint to validate the REANA specification of the workflow.

  • Changes create_workflow endpoint to populate workspace retention rules for the workflow.

  • Changes start_workflow endpoint to disallow restarting a workflow when retention rules are pending.

  • Changes API rate limiter error messages to be more verbose.

  • Changes workflow scheduler to allow defining the checks needed to assess whether the cluster can start new workflows.

  • Changes the Invenio dependencies to the latest versions.

  • Changes OAuth configuration to enable the new CERN SSO.

  • Changes to PostgreSQL 12.13.

  • Changes GitLab integration to also retrieve user’s projects that are in groups and subgroups.

  • Changes the base image of the component to Ubuntu 20.04 LTS and reduces final Docker image size by removing build-time dependencies.

  • Fixes issue when irregular number formats are passed to REANA_SCHEDULER_REQUEUE_COUNT configuration environment variable.

  • Fixes GitLab integration error reporting in case user exceeds CPU or Disk quota usage limits.

  • Fixes CERN OIDC authentication to possibly allow eduGAIN and social login users.

0.8.4 (2022-02-23)

  • Changes workflow scheduler to count number of workflow retries.

0.8.3 (2022-02-10)

  • Adds Kubernetes job memory limits validation before publishing workflow submission.

0.8.2 (2022-02-07)

  • Adds email validation to the user-create command used by the REANA administrators.

  • Adds workflow name validation to the create_workflow endpoint.

  • Changes /api/info endpoint to return a list of supported compute backends.

  • Changes /api/status endpoint to calculate the cluster health status based on the availablity instead of the usage.

0.8.1 (2021-11-29)

  • Changes quota-set command used by the REANA administrators to use the resource type along with a resource name for specifying the resource.

  • Changes email validation used in create-admin-user command by the REANA administrators to be more permissive.

0.8.0 (2021-11-22)

  • Adds users quota accounting.

  • Adds support for Snakemake workflow engine.

  • Adds include_progress and include_workspace_size query args to workflow list endpoint.

  • Adds workflow prioritization in the queue by complexity.

  • Adds priority and min_job_memory params to workflow submission publisher.

  • Adds Yadage workflow specification loading to start_workflow endpoint.

  • Adds a check in scheduler if at least one workflow job could be started in Kubernetes.

  • Adds configuration environment variable to set workflow scheduling policy (REANA_WORKFLOW_SCHEDULING_POLICY).

  • Adds configuration environment variable to set a timeout between consuming workflows (REANA_SCHEDULER_REQUEUE_SLEEP).

  • Adds configuration environment variable to set an API rate limiter (REANA_RATELIMIT_AUTHENTICATED_USER, REANA_RATELIMIT_GUEST_USER).

  • Adds new info endpoint allowing to retrieve information about cluster capabilities such as available workspaces.

  • Changes workflow execution consumer to receive only one message at a time.

  • Changes to PostgreSQL 12.8.

0.7.6 (2021-07-05)

  • Changes internal dependencies.

0.7.5 (2021-04-28)

  • Adds support for listing files using glob patterns.

  • Adds support for glob patterns and directory downloads, packaging the content into a zip file.

0.7.4 (2021-03-17)

  • Adds configuration to set a timeout between reana_ready checks. (REANA_SCHEDULER_SECONDS_TO_WAIT_FOR_REANA_READY)

  • Fixes start workflow endpoint to work with unspecified operational_options parameter

  • Fixes workflow scheduling bug in which failed worfklows would count as running, reaching REANA_MAX_CONCURRENT_BATCH_WORKFLOWS and therefore, blocking the job-submission queue.

0.7.3 (2021-02-03)

  • Adds optional email confirmation step after users sign up.

  • Changes email notifications with enriched instructions on how to grant user tokens.

0.7.2 (2020-11-24)

  • Changes rate limiting defaults to allow up to 20 connections per second.

  • Fixes minor code warnings.

0.7.1 (2020-11-10)

  • Fixes REANA <-> GitLab synchronisation for projects having additional external webhooks.

  • Fixes restarting of Yadage and CWL workflows.

  • Fixes conflicting kombu installation requirements by requiring Celery version 4.

  • Changes /api/you endpoint to include REANA server version information.

0.7.0 (2020-10-20)

  • Adds new endpoint to request user tokens.

  • Adds email notifications on relevant events such as user token granted/revoked.

  • Adds new templating system for notification email bodies.

  • Adds possibility to query logs for a single workflow step.

  • Adds endpoint to retrieve the workflow specification used for the workflow run.

  • Adds preview flag to download file endpoint.

  • Adds validation of submitted operational options before starting a workflow.

  • Adds possibility to upload empty files.

  • Adds new block size option to specify the type of units to use for disk size.

  • Adds a possibility to upload new workflow definitions before restarting a workflow.

  • Adds new command to generate status report for the REANA administrators; useful as a cronjob.

  • Adds user token management commands to grant and revoke user tokens.

  • Adds support for local user management.

  • Adds pinning of all Python dependencies allowing to easily rebuild component images at later times.

  • Fixes bug related to rescheduling deleted workflows.

  • Changes REANA_URL configuration variable to more precise REANA_HOSTNAME.

  • Changes workflow list endpoint response payload to include workflow progress information.

  • Changes import/export commands with respect to new user model fields.

  • Changes submodule installation in editable mode for live code updates for developers.

  • Changes pre-requisites to Invenio-Accounts 1.3.0 to support REST API.

  • Changes /api/me to /api/you endpoint due to conflict with Invenio-Accounts.

  • Changes base image to use Python 3.8.

  • Changes code formatting to respect black coding style.

  • Changes documentation to single-page layout.

0.6.1 (2020-05-25)

  • Upgrades REANA-Commons package using latest Kubernetes Python client version.

  • Pins Flask and Invenio dependencies to fix REANA 0.6 installation troubles.

0.6.0 (2019-12-20)

  • Fixes bug with big file uploads by using data streaming.

  • Adds user login endpoints using OAuth, currently configured to work with CERN SSO but extensible to use other OAuth providers such as GitHub, more in Invenio-OAuthClient.

  • Adds endpoints to integrate with GitLab (for retrieving user projects and creating/deleting webhooks).

  • Adds new endpoint /me to retrieve user information.

  • Improves security by allowing requests only with REANA_URL in the host header, avoiding host header injection attacks.

  • Initialisation logs moved from stdout to /var/log/reana-server-init-output.log.

0.5.0 (2019-04-23)

  • Adds new endpoint to compare two workflows. The output is a git like diff which can be configured to show differences at metadata level, workspace level or both.

  • Adds new endpoint to retrieve workflow parameters.

  • Adds new endpoint to query the disk usage of a given workspace.

  • Adds new endpoints to delete and move files whithin the workspace.

  • Adds new endpoints to open and close interactive sessions inside the workspace.

  • Workflow start does not send start requests to REANA Workflow Controller straight away, instead it will decide whether REANA can execute it or queue it depending on a set of conditions, currently it depends on the number of running jobs in the cluster.

  • Adds new administrator command to export and import all REANA users.

0.4.0 (2018-11-06)

  • Improves REST API documentation rendering.

  • Enhances test suite and increases code coverage.

  • Changes license to MIT.

0.3.1 (2018-09-07)

  • Harmonises date and time outputs amongst various REST API endpoints.

  • Pins REANA-Commons, REANA-DB and Bravado dependencies.

0.3.0 (2018-08-10)

  • Adds support of Serial workflows.

  • Adds API protection with API tokens.

0.2.0 (2018-04-19)

  • Adds support of Common Workflow Language workflows.

  • Adds support of specifying workflow names in REST API requests.

  • Improves error messages and information.

0.1.0 (2018-01-30)

  • Initial public release.

Contributing

Bug reports, issues, feature requests, and other contributions are welcome. If you find a demonstrable problem that is caused by the REANA code, please:

  1. Search for already reported problems.

  2. Check if the issue has been fixed or is still reproducible on the latest master branch.

  3. Create an issue, ideally with a test case.

If you create a pull request fixing a bug or implementing a feature, you can run the tests to ensure that everything is operating correctly:

$ ./run-tests.sh

Each pull request should preserve or increase code coverage.

License

MIT License

Copyright (C) 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025, 2026 CERN.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

In applying this license, CERN does not waive the privileges and immunities granted to it by virtue of its status as an Intergovernmental Organization or submit itself to any jurisdiction.

Authors

The list of contributors in alphabetical order: