The old create volume flow (v1) will still be offered as API endpoint for backwards compatibility. Only the metadata (file/text) fields might be removed from that endpoint.
Here is the concept: The v2 flow consists of multiple stages.
The user chooses a media type (image or video) and optionally selects a metadata file for upload. Advancing to the next step, the metadata file is uploaded and stored in a "pending volume". Relevant information from the file is extracted for the following steps (by the metadata parser https://github.com/biigle/core/issues/555).
In this step, the volume name is set (automatically filled if provided in the metadata but can be edited by the user). Also, the user selects a file source (remote, uploaded files, storage disk). There are different cases now (depending on the source and if a metadata file was supplied):
remote/no metadata: The user enters the URL and pastes a list of comma separated filenames (like in the flow v1). When the filenames are pasted, a list is generated showing the filenames in separate rows. Each file can be removed from the list with an "x" button.
remote/metadata: If available, the URL is prefilled from the metadata. Otherwise the user has to enter it. The filenames are prefilled (i.e. the filename list is displayed) from the metadata. The user can choose to ignore some files by removing them from the list (see above) or they can clear all files and enter a new comma-separated list of filenames.
uploaded files/no metadata: The user can select a directory or a list of files from the file browser. This will show a list of filenames (with the option to remove individual files) as described above.
uploaded files/metadata: The uploaded files are searched for a directory that contains all filenames from the metadata. This directory (or the subset of files) will be automatically selected. Otherwise there will be a warning that no suitable files could be found. Same than uploaded files/no metadata but there will be a warning if the selected files do not match the files from the metadata.
storage disk/no metadata: Same than uploaded files/no metadata.
storage disk/metadata: Same than uploaded files/no metadata but there will be a warning if the selected files do not match the files from the metadata.
If the metadata file includes the relevant information, the user can also select a checkbox to import annotations and/or a checkbox to import file labels in this step. Also, they can specify a handle or DOI for the volume (which is prefilled if the metadata file includes it).
The new volume will be created after this step. If annotations or file labels should be imported, the PendingVolume is kept and associated with the new volume ("Create volume and import annotations" or "Create volume and import image labels" button). Then there are the following additional steps. Otherwise, the PendingVolume is removed ("Create volume" button).
This step allows the user to specify which annotations should be imported (i.e. show available labels and allow the user to import only some of them).
This step allows the user to specify which image labels should be imported (i.e. show available labels and allow the user to import only some of them).
This step checks all labels that should be imported based on the selections in the previous two steps. If all labels to import already exist in the database (matched by their UUID), this step is automatically skipped. Otherwise, the user has the option to:
Select substitute labels from an existing label tree.
Add imported labels to an existing label tree.
Create a new label tree with the imported labels.
This step checks if all users that belong to the annotations/file labels to import already exists in the database (matched by their UUID). If not, the user who currently creates this volume has the option to:
Select a user from the database for each user of the metadata that was not found
Select themselves as the creator of the annotations/image labels
The new annotations and/or file labels will be created after this step. The existence of the labels chosen in step 5 has to be checked again. If some are missing, go back to that step.
When this process is complete, the new volume, files, annotations, file labels and labels are created. The "pending volume" is removed. The pending volume is also removed if this process is aborted or not finished within a week (or so).
When a pending volume exists, it is shown to the user who created it if they want to create another new volume for the same project. They can continue to create the pending volume or discard it (by creating a new one). They can not have multiple pending volumes for the same project.
This issue is about implementing a new flow to create volumes. In addition to what the current flow supports, it will support:
The old create volume flow (v1) will still be offered as API endpoint for backwards compatibility. Only the metadata (file/text) fields might be removed from that endpoint.
Here is the concept: The v2 flow consists of multiple stages.
The user chooses a media type (image or video) and optionally selects a metadata file for upload. Advancing to the next step, the metadata file is uploaded and stored in a "pending volume". Relevant information from the file is extracted for the following steps (by the metadata parser https://github.com/biigle/core/issues/555).
In this step, the volume name is set (automatically filled if provided in the metadata but can be edited by the user). Also, the user selects a file source (remote, uploaded files, storage disk). There are different cases now (depending on the source and if a metadata file was supplied):
remote/no metadata: The user enters the URL and pastes a list of comma separated filenames (like in the flow v1).
When the filenames are pasted, a list is generated showing the filenames in separate rows. Each file can be removed from the list with an "x" button.remote/metadata: If available, the URL is prefilled from the metadata. Otherwise the user has to enter it. The filenames are prefilled (i.e. the filename list is displayed) from the metadata.
The user can choose to ignore some files by removing them from the list (see above) or they can clear all files and enter a new comma-separated list of filenames.uploaded files/no metadata: The user can select a directory or a list of files from the file browser.
This will show a list of filenames (with the option to remove individual files) as described above.uploaded files/metadata:
The uploaded files are searched for a directory that contains all filenames from the metadata. This directory (or the subset of files) will be automatically selected. Otherwise there will be a warning that no suitable files could be found.Same than uploaded files/no metadata but there will be a warning if the selected files do not match the files from the metadata.storage disk/no metadata: Same than uploaded files/no metadata.
storage disk/metadata: Same than uploaded files/no metadata but there will be a warning if the selected files do not match the files from the metadata.
If the metadata file includes the relevant information, the user can also select a checkbox to import annotations and/or a checkbox to import file labels in this step. Also, they can specify a handle or DOI for the volume (which is prefilled if the metadata file includes it).
The new volume will be created after this step. If annotations or file labels should be imported, the PendingVolume is kept and associated with the new volume ("Create volume and import annotations" or "Create volume and import image labels" button). Then there are the following additional steps. Otherwise, the PendingVolume is removed ("Create volume" button).
This step allows the user to specify which annotations should be imported (i.e. show available labels and allow the user to import only some of them).
This step allows the user to specify which image labels should be imported (i.e. show available labels and allow the user to import only some of them).
This step checks all labels that should be imported based on the selections in the previous two steps. If all labels to import already exist in the database (matched by their UUID), this step is automatically skipped. Otherwise, the user has the option to:
Select substitute labels from an existing label tree.
Add imported labels to an existing label tree.
Create a new label tree with the imported labels.
This step checks if all users that belong to the annotations/file labels to import already exists in the database (matched by their UUID). If not, the user who currently creates this volume has the option to:
Select a user from the database for each user of the metadata that was not found
Select themselves as the creator of the annotations/image labels
The new annotations and/or file labels will be created after this step. The existence of the labels chosen in step 5 has to be checked again. If some are missing, go back to that step.
When this process is complete, the new volume, files, annotations, file labels and labels are created. The "pending volume" is removed. The pending volume is also removed if this process is aborted or not finished within a week (or so).
When a pending volume exists, it is shown to the user who created it if they want to create another new volume for the same project. They can continue to create the pending volume or discard it (by creating a new one). They can not have multiple pending volumes for the same project.