Using an ad-hoc GUI the user can import thier projects from another deployment.
Prerequisits:
user must have an account in bouth source and destination deployments
user must authenticate with his credentials from source inside the destination deployment (this generates tokesn for the purpose of importing projects)
Chnages to oSPARC:
create endpoint for authenticating the user in another deployment
create endpoint for listing projects available to the user (maybe we can reuse soemething?)
create endpoint to start a copy (lock project): provides "project data" + "tokens to copy data from s3"
sumbit a job that "imports" the project: first sync data then insert project in db, if it fails remove data.
create endpoint for signal copy operation is done (unlocks project)
PROS:
not very complex, we rely on already existing tools and just generate a few ne API endpoints
potentially can be used internally to make a copy of an existing project (target the same deployment)
avoids creating a "data model" for exporting and importing user data by rclone copy S3 to S3
CONS:
user does not get access to their data (they can only move it from deployment A to deployment B)
(2) Archiving
Generate an archive containing project data and data stored in all nodes.
Prerequisits:
user must have an account in both source and desitnation deployments
user must have enough disk space to download the archive to his computer
Changes to oSPARC:
create endpoint for starting the export procedure
background job that creates the archive:
donwload files and put them in an archive envtually compressing them + packaging the data model for the porject
upload the arhcive to S3 (with an expiration)
notify user (via email?) that the arhvie is available for download
solid upload process that is able to resume (require backend/FE coordination)
split file into chunks
retry if chunk fails to upload
put chunks together in a unique file
import process (once file is available start import)
check archive validity (nobody tamperred with it)
extract data from the archive and upload to S3 (rollback on error)
insert project in DB
PROS:
user has phisical copy of the data, by opening the archive he could extract a single file
CONS:
requires a third party computer (the user's) to download the arhive and upload the archive
uses two extra step form solution (1): archive creation and archive extraction
Based on the working group https://github.com/ITISFoundation/osparc-ops-environments/issues/672 we decided we will investigate these 3 options:
(1) Importing from target deployment
Using an ad-hoc GUI the user can import thier projects from another deployment.
Prerequisits:
Chnages to oSPARC:
PROS:
CONS:
(2) Archiving
Generate an archive containing project data and data stored in all nodes.
Prerequisits:
Changes to oSPARC:
PROS:
CONS:
(3) Migration
...
Prerequisits:
Changes to oSPARC:
PROS:
CONS: