search

NDAR / nda-tools

Python package for interacting with NDA web services. Used to validate, submit, and download data to and from NDA.
MIT License
48 stars 22 forks source link

Submission of data with custom scope fails with "Unrecognized column name" #34

Closed jacob-lee closed 3 years ago

jacob-lee commented 3 years ago

Submission of instruments with custom scope fails using the tool, but does not fail when using the online tool. The error log indicates that the columns of the submitted CSV were not recognized as belonging to the data dictionary, e.g.

54fc58-742c-4066-86e9-0eaf0d151ff2,CompleteWithErrors,2021-07-07T11:13:24.900-0400,unrecognizedColumnName,scid_5_pd1,Data column names must all match with the structure element_names or aliases. Unrecognized column names are not permitted in your data file.,

When submitting data with custom scope using nda tools, you enter in the optional --scope argument, which documentation indicates has type string. In particular, it doesn't indicate that it is an integer or a sequence like a list or tuple.

I ran with pdbdebugger and discovered that in the process of parsing the arguments to the validation tool, the scope argument is treated as a list or other subscriptable data structure. In particular, my scope (which is the collection id) went from "5XXXX" to "5", which is possible because in Python strings are subscriptable. To confirm that this was indeed the problem, I modified my local copy to not subscript the scope argument, and I was able to submit the data structure successfully.

gregmagdits commented 3 years ago

Hello, We have an open PR with the fix for this issue. We plan on deploying this to master today with version 0.2.7

gregmagdits commented 3 years ago

Deployed fix from PR-31 into master - closing ...