Upload layers to existing datasets

[hotzenklotz]

Detailed Description

It would be great if I could upload a new color/segmentation layer to an existing dataset in WK. It would be great if one could do that through the UI and WK-Libs.

Currently, one would need either access to the file system to add a layer or re-upload a combination of raw data + new layer.

Context

The web uploader already works great but we have users who upload a dataset layer per layer without realizing their "mistake". There is no way for them to add a layer later.

The WK-Libs offer linking feature but that is also very hard to explain and symlinks can be a pain.

• [ ] Specific to long-running jobs (set jobsEnabled=true in application.conf)
• [ ] Specific to webknossos.org (set isDemoInstance=true in application.conf)

[fm3]

One thing that is similar would be layers_to_link concept in uploads, as offered by the backend. With that, you don’t have to reupload layers that are already present in wk, and can compose datasets by specifying the existing layers during the upload. It will still create a new dataset, rather than mutate an existing one, but the data is not duplicated on disk, but linked in.

I think the libs already do expose this, though I’m not super familiar with that.

The frontend could in theory also use this.

However, it might still also make sense to actually mutate existing datasets by adding new layers, as the issue description states.

[philippotto]

The front-end part will be (partly) covered by https://github.com/scalableminds/webknossos/pull/7395. The use-case there is focussed on transformations, but the UI would also allow to combine layers (compose datasets) without any transforms.

However, it might still also make sense to actually mutate existing datasets by adding new layers, as the issue description states.

True. However, I'd wait for #7395 until we decide next steps.

[kabilar]

+1

Hi Scalable Minds team, it would definitely be helpful to be able to add additional layers to an existing dataset on the front-end. Thank you.

cc @aaronkanzer

[philippotto]

Hi @kabilar,

this is currently possible by uploading the additional layers as a new dataset and then combining that new dataset with the existing dataset. The combination part can be done within the "Compose From Existing Datasets" tab within the Import UI. I know, the workflow could be more streamlined, but hopefully this approach already helps you.