search

OpenNeuroOrg / openneuro

A free and open platform for analyzing and sharing neuroimaging data
https://openneuro.org/
MIT License
112 stars 39 forks source link

Unable to upload dataset with CLI after passing bids-validator #2742

Closed koreywylie closed 1 year ago

koreywylie commented 1 year ago

Unable to upload dataset using command line interface v4.14.0, despite this dataset passing bids-validator with minor warnings, command-line output attached. openneuro_upload_output.txt bids-validator_output.txt

nellh commented 1 year ago

Can you provide the version of Node.js and what OS you're uploading from?

koreywylie commented 1 year ago

OSX 10.15.7

$ npm list -g /usr/local/lib ├── @@. ├── @. ├── @. └── @.

Thanks & best, --Korey--

From: Nell Hardcastle @.> Sent: Saturday, December 24, 2022 1:15 PM To: OpenNeuroOrg/openneuro @.> Cc: Wylie, Korey @.>; Author @.> Subject: Re: [OpenNeuroOrg/openneuro] Unable to upload dataset with CLI after passing bids-validator (Issue #2742)

[External Email - Use Caution]

Can you provide the version of Node.js and what OS you're uploading from?

— Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOpenNeuroOrg%2Fopenneuro%2Fissues%2F2742%23issuecomment-1364579315&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7Ca844df7130294bb14dc608dae5ebaa2a%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638075097615000800%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XIAwPd%2Fj0PMjvKx%2FeeRnoIVOuRHtzz%2BfMMTUQ6TB9k0%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHVRDVAOV7PWUM5QJIG53WLWO5KXXANCNFSM6AAAAAATIB4XOY&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7Ca844df7130294bb14dc608dae5ebaa2a%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638075097615000800%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9RNnWRCl3o5YwknZJiselssp7f2%2BMv%2BLVKNjzy4JbSo%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>

Arshitha commented 1 year ago

Hi I'm having the same issue.

I'm running the command from HPC system with 

Linux version 3.10.0-1160.80.1.el7.x86_64 (mockbuild@x86-vm-41.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) )

and software versions as follows: 

openneuro v4.14.0
node.js v18.12.1

I'm trying to update an existing dataset (ds004215) but it gives me the following stdout:

[arshithab@biowulf v1.0.2_changes]$ openneuro upload -d ds004215 -i -n --ignoreSubjectConsistency .
Adding files to "ds004215"
bids-validator@1.9.9
        Summary:                  Available Tasks:        Available Modalities: 
        512 Files, 11.01MB                                MEG                   
        66 - Subjects                                                           
        1 - Session                                                             

        If you have any questions, please post on https://neurostars.org/tags/bids.

Checking remote files...
(node:110534) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
=======================================================================
512 files to be uploaded with a total size of 11.0 MB
? Begin upload? Yes
=======================================================================
Starting a new upload (58924465) to dataset: 'ds004215'
d. 11 abort listeners added to [AbortSignal]. Use events.setMaxListeners() to increase limittenersExceededWarning: Possible EventTarget memory leak detecte
Retrying upload for dataset_description.json: TypeError: fetch failed
Retrying upload for dataset_description.json: TypeError: Cannot construct a Request with a Request object that has already been used.
Retrying upload for dataset_description.json: TypeError: Cannot construct a Request with a Request object that has already been used.
Retrying upload for dataset_description.json: TypeError: Cannot construct a Request with a Request object that has already been used.
Retrying upload for dataset_description.json: TypeError: Cannot construct a Request with a Request object that has already been used.
Retrying upload for dataset_description.json: TypeError: Cannot construct a Request with a Request object that has already been used.
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-airpuff_run-01_channels.tsv: TypeError: fetch failed
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-airpuff_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-airpuff_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-airpuff_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-airpuff_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: fetch failed
t that has already been used.47/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request objec
t that has already been used.47/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request objec
t that has already been used.47/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request objec
t that has already been used.47/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request objec
t that has already been used.47/ses-01/meg/sub-ON02747_ses-01_task-artifact_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request objec
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-gonogo_run-01_channels.tsv: TypeError: fetch failed
that has already been used.2747/ses-01/meg/sub-ON02747_ses-01_task-gonogo_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object 
that has already been used.2747/ses-01/meg/sub-ON02747_ses-01_task-gonogo_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object 
that has already been used.2747/ses-01/meg/sub-ON02747_ses-01_task-gonogo_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object 
that has already been used.2747/ses-01/meg/sub-ON02747_ses-01_task-gonogo_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object 
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: fetch failed
bject that has already been used.es-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request o
bject that has already been used.es-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request o
bject that has already been used.es-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request o
bject that has already been used.es-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request o
bject that has already been used.es-01/meg/sub-ON02747_ses-01_task-haririhammer_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request o
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-movie_run-01_channels.tsv: TypeError: fetch failed
hat has already been used.02747/ses-01/meg/sub-ON02747_ses-01_task-movie_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object t
hat has already been used.02747/ses-01/meg/sub-ON02747_ses-01_task-movie_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object t
hat has already been used.02747/ses-01/meg/sub-ON02747_ses-01_task-movie_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object t
hat has already been used.02747/ses-01/meg/sub-ON02747_ses-01_task-movie_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object t
Retrying upload for sub-ON02747/ses-01/meg/sub-ON02747_ses-01_task-oddball_run-01_channels.tsv: TypeError: fetch failed
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-oddball_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
 that has already been used.747/ses-01/meg/sub-ON02747_ses-01_task-oddball_run-01_channels.tsv: TypeError: Cannot construct a Request with a Request object
....
....

It goes on for each file like this (attached is the full output text) and the program exits with the following message:

=======================================================================
Upload Complete
To publish the update go to https://openneuro.org/datasets/ds004215 and create a new snapshot
=======================================================================

ds004215_openneuro_upload_stdout.txt

nellh commented 1 year ago

Thank you for the bug reports, we're testing a fix for this and it will be released soon.

If you would like to try the fix, it's available on npm as version 4.14.1-alpha.0 and can be installed with npm install @openneuro/cli@4.14.1-alpha.0.

nellh commented 1 year ago

This fix has been released in OpenNeuro 4.14.1, please let us know if you continue to see issues.

Arshitha commented 1 year ago

Thanks @nellh! that worked :)

koreywylie commented 1 year ago

Worked for me as well, thanks for resolving this issue, @nellhhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnellh&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7C8c59f4b33a09472daf4908daef6bd784%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638085543753625861%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=k%2FHzWf2nyWM4d8ziZ47wZW7LOxJXzu%2Fu2%2F35XsWyNzg%3D&reserved=0!

From: Arshitha Basavaraj @.> Sent: Thursday, January 5, 2023 3:26 PM To: OpenNeuroOrg/openneuro @.> Cc: Wylie, Korey @.>; Author @.> Subject: Re: [OpenNeuroOrg/openneuro] Unable to upload dataset with CLI after passing bids-validator (Issue #2742)

[External Email - Use Caution]

Thanks @nellhhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnellh&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7C8c59f4b33a09472daf4908daef6bd784%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638085543753625861%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=k%2FHzWf2nyWM4d8ziZ47wZW7LOxJXzu%2Fu2%2F35XsWyNzg%3D&reserved=0! that worked :)

— Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOpenNeuroOrg%2Fopenneuro%2Fissues%2F2742%23issuecomment-1372871396&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7C8c59f4b33a09472daf4908daef6bd784%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638085543753625861%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xwF4VtJkj%2B0VYToHqx5Hg5YAV3CNxsLUmV%2BgVMrTqA0%3D&reserved=0, or unsubscribehttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHVRDVA3OKOJQPOEFRDDH4LWQ5C77ANCNFSM6AAAAAATIB4XOY&data=05%7C01%7Ckorey.wylie%40cuanschutz.edu%7C8c59f4b33a09472daf4908daef6bd784%7C563337caa517421aaae01aa5b414fd7f%7C0%7C0%7C638085543753625861%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7PJjn6tLzbpVFCgiERIsACeDCY8LhzY%2BYW2%2FomgYcro%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>