artefactual-labs / umcu-uploader

UMCU Uploader
Apache License 2.0
1 stars 1 forks source link

division-specific folders in the data directory are not used #97

Open sandwitch opened 1 year ago

sandwitch commented 1 year ago

I was told that the uploaders places the directories uploaded at first in the division folders in the datadirectory, however it places everything in the root of the datadirectoy. Is this a miscobfiguration on our part or a bug?

Lising current situation:

ls -l /srv/Park/ total 1024 drwxr-xr-x. 2 uploader uploader 0 9 feb 09:38 _1 drwxr-xr-x. 2 uploader uploader 0 10 feb 13:49 _2 -rw-r-----. 1 uploader uploader 4240 22 feb 2022 2022-02-22_Algemeen.txt drwxr-xr-x. 2 uploader uploader 0 9 mrt 10:58 _3 drwxr-xr-x. 2 uploader uploader 0 27 mrt 12:20 _4 drwxr-xr-x. 2 uploader uploader 0 27 mrt 14:55 _5 drwxr-xr-x. 2 uploader uploader 0 27 mrt 14:57 _6 drwxr-xr-x. 2 uploader uploader 0 23 feb 2022 AAAA drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 beeld drwxr-xr-x. 2 uploader uploader 0 9 feb 09:26 cc drwxr-xr-x. 2 uploader uploader 0 28 mrt 08:49 datavarse_test drwxr-xr-x. 2 uploader uploader 0 9 mrt 2022 dit drwxr-xr-x. 2 uploader uploader 0 28 mrt 11:30 folder2 drwxr-xr-x. 2 uploader uploader 0 9 feb 10:01 geckodriver-v0.32.0-linux64 drwxr-xr-x. 2 uploader uploader 0 9 feb 11:51 geckodriver-v0.32.0-linux64_1 drwxr-xr-x. 2 uploader uploader 0 27 mrt 13:08 geckodriver-v0.32.0-linux64_2 drwxr-xr-x. 2 uploader uploader 0 9 feb 11:25 G_Output drwxr-xr-x. 2 uploader uploader 0 14 feb 2022 her drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 hl drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 hs drwxr-xr-x. 2 uploader uploader 0 9 feb 11:16 igd drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 jc drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 kind drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 lab drwxr-xr-x. 2 uploader uploader 0 9 feb 11:11 Oefeningen drwxr-xr-x. 2 uploader uploader 0 9 feb 15:52 Oefeningen_1 drwxr-xr-x. 2 uploader uploader 0 9 feb 16:12 Oefeningen_2 drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 owc drwxr-xr-x. 2 uploader uploader 0 9 feb 10:37 temp drwxr-xr-x. 2 uploader uploader 0 27 mrt 10:49 test_am_uploader_20230327 drwxr-xr-x. 2 uploader uploader 0 27 mrt 12:22 test_am_uploader_20230327_1 drwxr-xr-x. 2 uploader uploader 0 27 mrt 14:55 test_am_uploader_20230327_2 drwxr-xr-x. 2 uploader uploader 0 27 mrt 15:51 test_am_uploader_20230327_3 drwxr-xr-x. 2 uploader uploader 0 29 mrt 10:23 test_am_uploader_20230327_4 drwxr-xr-x. 2 uploader uploader 0 9 feb 14:05 Test data drwxr-xr-x. 2 uploader uploader 0 9 feb 15:12 Test data_1 drwxr-xr-x. 2 uploader uploader 0 10 feb 12:43 Test data_2 drwxr-xr-x. 2 uploader uploader 0 10 feb 13:50 Test data_3 drwxr-xr-x. 2 uploader uploader 0 13 feb 10:51 Test data_4 drwxr-xr-x. 2 uploader uploader 0 9 mrt 10:29 Test data_5 drwxr-xr-x. 2 uploader uploader 0 27 mrt 14:56 Test data_6 drwxr-xr-x. 2 uploader uploader 0 9 feb 13:27 test_folder_1 drwxr-xr-x. 2 uploader uploader 0 13 feb 11:07 test_folder_1_1 drwxr-xr-x. 2 uploader uploader 0 14 feb 11:58 test_folder_1_2 drwxr-xr-x. 2 uploader uploader 0 9 mrt 10:59 test_folder_1_3 drwxr-xr-x. 2 uploader uploader 0 27 mrt 14:36 test_folder_1_4 drwxr-xr-x. 2 uploader uploader 0 28 mrt 14:20 test_folder_1_5 drwxr-xr-x. 2 uploader uploader 0 29 mrt 10:45 test_folder_1_6 drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 vb drwxr-xr-x. 2 uploader uploader 0 10 aug 2021 vf

For clarification files in the division folders are from before the uploader install:

for DIR in beeld cc dit her hl hs igd jc kind lab owc vb vf ; do ls -l /srv/Park/${DIR} ; done total 0 total 0 total 0 drwxr-xr-x. 2 uploader uploader 0 16 feb 2022 22_Athero drwxr-xr-x. 2 uploader uploader 0 16 feb 2022 22_Beeld drwxr-xr-x. 2 uploader uploader 0 3 mrt 2022 22_COVID_19 drwxr-xr-x. 2 uploader uploader 0 15 feb 2022 22_Dermatologie total 0 drwxr-xr-x. 2 uploader uploader 0 21 jan 2021 22_YOUth_spde total 0 total 0 total 0 drwxr-xr-x. 2 uploader uploader 0 9 feb 11:16 test_folder_1 total 0 total 0 total 0 total 0 total 0 total 0

sandwitch commented 1 year ago

Our config:

umcu_uploader.yaml.txt

mcantelon commented 1 year ago

Hi @sandwitch ! Uploaded research folders don't currently get put into division-specific subdirectories in the data directory. Only Archivematica transfer source directories can currently be configured to be division-specific.

joel-simpson commented 1 year ago

I think that makes sense... when you first upload a dataset to the UMCU uploader, you don't know what division it should go into until the user has filled in the metadata form.