NDAR / nda-tools

Python package for interacting with NDA web services. Used to validate, submit, and download data to and from NDA.
MIT License
48 stars 22 forks source link

Adding a Text File to Uploads #37

Closed JBarsotti closed 8 months ago

JBarsotti commented 2 years ago

Hi Greg!

Is there any way to add a text file with an explanation of some data elements to an NDA upload? I keep getting a validation error when I try to include it, so don't know if it's possible.

Thanks!

John

gregmagdits commented 2 years ago

Hi John, Can you clarify - are you trying to add new data elements to the data-structure? Also, what is the validation error that you are getting?

JBarsotti commented 2 years ago

Hi Greg,

Thanks so much for your reply! I hope you are enjoying the weekend.

The upload I am trying to do contains new subject data, as well as corrections to the subject data submitted in a previous upload. I would like to add a text file outlining the changes to previously uploaded subject data (i.e., something like a ReadMe file). I keep getting an error saying that the text file does not pass validation, which makes sense since it is a new data element. Is there any way to upload it still?

Also, I do seem to have a problem where when I remove the text file and just upload files that were validated, the image03.csv file and the ndar_subject01.csv files don't seem to be included in the upload. I think it's because I am making a mistake in the file paths. When I enter the directory of "associated files," is that the directory where imaging files are contained only? The image03.csv and ndar_subject01.csv files are in a separate directory from the imaging files, but maybe this is wrong.

Thanks again for all your help,

John

gregmagdits commented 2 years ago

Hi John, Instead of uploading a file that contains a summary of what was corrected, NDA usually just archives your previous file-upload so that only the corrected data is publicly accessible. That being said, you could request a new data-element be added to a structure by contacting your collection's data-curator by opening a ticket at https://ndar.zendesk.com/agent. Part of the job of the data-curator is to avoid adding data-elements un-necessarily to structures, so if I were your curator and if you still wanted to add additional documentation to your upload, I would propose that you upload your README to your collection's supporting documentation instead of creating a new element (see supporting documentation section on your collection dashboard https://nda.nih.gov/edit_collection.html?id=2810).

You are correct in that the directory of the "associated files," is the directory where imaging files are contained only. In order to upload the ndar_subject01.csv and image03.csv files, you would need to use the following command syntax :

vtcmd -b /path/to/ndar_subject01.csv /path/to/image03.csv -l /path/to/associated_files

The above command will create one submission containing both the ndar_subject01 and image03 structures. If the above syntax isn't working as expected, or if I misunderstood the question, we can hop on a zoom call to troubleshoot the issue.

Thanks

JBarsotti commented 2 years ago

Hi Greg,

Thanks so much for your replies! I will give that command a try and see if it uploads all the files. If not, I'd love to talk to you over Zoom.

Thanks,

John

JBarsotti commented 2 years ago

Hi Greg,

I tried doing the upload using the command you gave me but am still encountering the same problem. The MRI files seem to be getting uploaded, but not the image03.csv or ndar_subject01.csv files. What's weird is that the directory in which these files are contained "Upload" is uploaded, but when I look in it, the only file there is the zip file for the "bold rest" experiment.

I'd love to talk to you about it if you ever have a few minutes.

Thanks,

John

gregmagdits commented 2 years ago

Hi John, I just sent you an email with a MS teams invite for tomorrow morning. If you didn't get it, let me know. Otherwise, ill see you tomorrow to troubleshoot the issue.