kousu
commented
4 years ago I tried it with arabic characters and got a different problem: it accepted the upload but the WebUI says "Incomplete Upload" and everytime I try again it reuploads the same two files -- one of which is the arabic text, but the other which is just one of the normal ones in the source dataset.
$ openneuro download -s 1.0.0 ds002982 ds
Downloading ".bidsignore" - size 33 bytes
Downloading "CHANGES" - size 65 bytes
Downloading "README" - size 6 bytes
Downloading "dataset_description.json" - size 250 bytes
Downloading "participants.json" - size 389 bytes
Downloading "participants.tsv" - size 934 bytes
Downloading "sub-chiba750/anat/sub-chiba750_T2star.json" - size 788 bytes
Downloading "sub-glen/anat/sub-glen_T2star.json" - size 1495 bytes
Downloading "sub-chiba750/anat/sub-chiba750_T2star.nii.gz" - size 5543775 bytes
Downloading "sub-glen/anat/sub-glen_T2star.nii.gz" - size 1446307 bytes
$ grep -ri git ds | grep -v Binary
ds/CHANGES:{"error": "git object command exited with non-zero return code (1)"}
ds/sub-chiba750/anat/sub-chiba750_T2star.json:{"error": "git object command exited with non-zero return code (1)"}
ds/participants.json:{"error": "git object command exited with non-zero return code (1)"}
$ openneuro download -s 1.0.0 ds002982 ds # redownload because of #1693
Skipping present file ".bidsignore"
Downloading "CHANGES" - size 65 bytes
Skipping present file "README"
Skipping present file "dataset_description.json"
Downloading "participants.json" - size 389 bytes
Skipping present file "participants.tsv"
Downloading "sub-chiba750/anat/sub-chiba750_T2star.json" - size 788 bytes
Skipping present file "sub-glen/anat/sub-glen_T2star.json"
Skipping present file "sub-chiba750/anat/sub-chiba750_T2star.nii.gz"
Skipping present file "sub-glen/anat/sub-glen_T2star.nii.gz"
$ grep -ri git ds | grep -v Binary
$ mkdir ds/derivatives
$ echo cavabienaller > ds/derivatives/filter_علم الأعصاب.txt
$ openneuro upload ds
? This will create a new dataset, continue? Yes
bids-validator@1.5.3
Summary: Available Tasks: Available Modalities:
5 Files, 5.29MB T2star
1 - Subject
1 - Session
If you have any questions, please post on https://neurostars.org/tags/bids.
"ds002988" created with label "ds"
=======================================================================
Upload Complete
To publish your dataset go to https://openneuro.org/datasets/ds002988
=======================================================================
Transferring "dataset_description.json" - 100% complete
Transferring ".bidsignore" - 100% complete
Transferring "CHANGES" - 100% complete
Transferring "README" - 100% complete
Transferring "participants.json" - 100% complete
Transferring "participants.tsv" - 100% complete
Transferring "derivatives/filter_علم" - 100% complete
Transferring "sub-chiba750/anat/sub-chiba750_T2star.json" - 100% complete
Transferring "sub-chiba750/anat/sub-chiba750_T2star.nii.gz" - 14% complete (4.5 MB remaining)
$ openneuro upload -d ds002988 ds
Adding files to "ds002988"
bids-validator@1.5.3
Summary: Available Tasks: Available Modalities:
5 Files, 5.29MB T2star
1 - Subject
1 - Session
If you have any questions, please post on https://neurostars.org/tags/bids.
Skipping existing file - ".bidsignore"
Skipping existing file - "CHANGES"
Skipping existing file - "README"
Skipping existing file - "dataset_description.json"
Skipping existing file - "participants.json"
Skipping existing file - "participants.tsv"
Skipping existing file - "sub-chiba750/anat/sub-chiba750_T2star.json"
=======================================================================
Upload Complete
To publish the update go to https://openneuro.org/datasets/ds002988 and create a new snapshot
=======================================================================
Transferring "derivatives/filter_علم" - 100% complete
Transferring "sub-chiba750/anat/sub-chiba750_T2star.nii.gz" - 14% complete (4.5 MB remaining)
$ openneuro upload -d ds002988 ds
Adding files to "ds002988"
bids-validator@1.5.3
Summary: Available Tasks: Available Modalities:
5 Files, 5.29MB T2star
1 - Subject
1 - Session
If you have any questions, please post on https://neurostars.org/tags/bids.
Skipping existing file - ".bidsignore"
Skipping existing file - "CHANGES"
Skipping existing file - "README"
Skipping existing file - "dataset_description.json"
Skipping existing file - "participants.json"
Skipping existing file - "participants.tsv"
Skipping existing file - "sub-chiba750/anat/sub-chiba750_T2star.json"
=======================================================================
Upload Complete
To publish the update go to https://openneuro.org/datasets/ds002988 and create a new snapshot
=======================================================================
Transferring "derivatives/filter_علم" - 100% complete
Transferring "sub-chiba750/anat/sub-chiba750_T2star.nii.gz" - 13% complete (4.6 MB remaining)
Describe the bug
A filename with an "unusual" (i.e. non-american) characters in it will be mis-parsed by
mutation uploadFiles().To Reproduce
openneuro download -s 1.0.0 ds002982 ds# but watch out for #1693; verify the download was clean bygrep -ri git ds | grep -v Binaryopenneuro upload ds# accept the 'create a new dataset' promptExpected behavior
Filenames should respect the names I upload them under.
Screenshots
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
These files don't seem to be deletable.
The source of the weird '"'s that split across directory hierarchies is
git ls-tree. It handles unprintable characters by quoting the whole path:I tried adding a file with an embedded tab in it, and the pubsub notification that came through updated the list with the expected filename. However, upon reloading the dataset page it too had '"'s around its path.
Other unusual characters, like
\bor\aor tabs, cause a similar issue.