search

dandi / dandi-archive

DANDI API server and Web app
https://dandiarchive.org
13 stars 10 forks source link

errors out with 500 upon creation of 000004 #94

Closed yarikoptic closed 3 years ago

yarikoptic commented 3 years ago

All other dandisets were created ok, so I guess it is something in metadata? heroku dashboard is just hanging for me ATM, so cannot even try to see what is in the logs

$> python -m pdb /home/yoh/proj/dandi/dandi-cli/tools/migrate-dandisets.py https://api.dandiarchive.org/api  $DANDI_API_TOKEN 000004 --delete-extant
> /home/yoh/proj/dandi/dandi-cli/tools/migrate-dandisets.py(2)<module>()
-> import click
(Pdb) r
Creating Dandiset 000004
--Return--
> /home/yoh/proj/dandi/dandi-cli/tools/migrate-dandisets.py(37)<module>()->None
-> main()
(Pdb) c
Traceback (most recent call last):
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/pdb.py", line 1704, in main
    pdb._runscript(mainpyfile)
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/pdb.py", line 1573, in _runscript
    self.run(statement)
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/bdb.py", line 580, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "/home/yoh/proj/dandi/dandi-cli/tools/migrate-dandisets.py", line 37, in <module>
    main()
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/yoh/miniconda3/envs/dandi-devel/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/yoh/proj/dandi/dandi-cli/tools/migrate-dandisets.py", line 31, in main
    client.create_dandiset(
  File "/home/yoh/proj/dandi/dandi-cli/dandi/dandiapi.py", line 475, in create_dandiset
    return self.post("/dandisets/", json={"name": name, "metadata": metadata})
  File "/home/yoh/proj/dandi/dandi-cli/dandi/dandiapi.py", line 185, in post
    return self.send_request(
  File "/home/yoh/proj/dandi/dandi-cli/dandi/dandiapi.py", line 149, in send_request
    raise requests.HTTPError(msg, response=result)
requests.exceptions.HTTPError: Error 500 while sending POST request to https://api.dandiarchive.org/api/dandisets/
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /home/yoh/proj/dandi/dandi-cli/dandi/dandiapi.py(149)send_request()
-> raise requests.HTTPError(msg, response=result)
(Pdb) p url
'https://api.dandiarchive.org/api/dandisets/'
(Pdb) p json
{'name': 'A NWB-based dataset and processing pipeline of human single-neuron activity during a declarative memory task', 'metadata': {'access': [{'status': 'dandi:Open'}], 'relatedResource': [{'identifier': 'DOI:10.17605/OSF.IO/HV7JA', 'name': 'A NWB-based Da
taset and Processing Pipeline of Human Single-Neuron Activity During a Declarative Memory Task', 'repository': 'Open Science Framework', 'url': 'https://osf.io/hv7ja/', 'relation': 'dandi:IsDerivedFrom'}, {'identifier': 'DOI:10.1038/s41597-020-0415-9', 'relat
ion': 'dandi:IsDescribedBy', 'url': 'https://www.nature.com/articles/s41597-020-0415-9'}], 'about': [{'name': 'Medial Temporal Lobe'}], 'contributor': [{'roleName': ['dandi:Author', 'dandi:ContactPerson', 'dandi:DataCurator', 'dandi:DataManager', 'dandi:Forma
lAnalysis', 'dandi:Investigation', 'dandi:Maintainer', 'dandi:Methodology', 'dandi:ProjectLeader', 'dandi:ProjectManager', 'dandi:ProjectMember', 'dandi:Researcher', 'dandi:Software', 'dandi:Validation', 'dandi:Visualization'], 'identifier': '0000-0003-0161-4
007', 'email': 'nandc10@gmail.com', 'name': 'Chandravadia, Nand', 'affiliation': [{'name': 'Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, '
schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:Methodology', 'dandi:ProjectMember', 'dandi:Software', 'dandi:Validation'], 'email': 'liang134@mail.chapman.edu', 'name': 'Liang, Dehua', 'affiliation': [{'name': 'Institute for Interdisciplinary Bra
in and Behavioral Sciences, Crean College of Health and Behavioral Sciences, Schmid College of Science and Technology, Chapman University, Orange, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Pe
rson'}, {'roleName': ['dandi:Author', 'dandi:DataCollector', 'dandi:ProjectMember', 'dandi:Validation'], 'identifier': '0000-0002-4319-7689', 'email': 'Andrea.Schjetan@uhnresearch.ca', 'name': 'Schjetnan, Andrea Gomez Palacio', 'affiliation': [{'name': 'Kremb
il Brain Institute, Toronto Western Hospital, Toronto, Canada', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:DataCurator', 'dandi:ProjectMember', 'dandi:Vali
dation'], 'identifier': '0000-0002-9207-7069', 'email': 'april.carlson@tufts.edu', 'name': 'Carlson, April', 'affiliation': [{'name': 'Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Or
ganization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:DataCollector', 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'mailyscm.faraut@gmail.com', 'name': 'Faraut, Mailys', 'affiliation': [{'name': 'Depart
ment of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:ProjectMember', 'dandi:Validation'], '
email': 'Jeffrey.Chung@cshs.org', 'name': 'Chung, Jeffrey M.', 'affiliation': [{'name': 'Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schema
Key': 'Person'}, {'roleName': ['dandi:Author', 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'Chrystal.Reed@csmc.edu', 'name': 'Reed, Chrystal M.', 'affiliation': [{'name': 'Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA', '
includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:Software', 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'ben.dichter@gmail.com', 'name': 'Dichter, Ben', 
'affiliation': [{'name': 'Biological Systems & Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'name': 'Department of Neurosurgery, Stanford University, Stanford, CA, 
USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:Conceptualization', 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'maoz.uri@gmail.com', 'name': 'Ma
oz, Uri', 'affiliation': [{'name': 'Institute for Interdisciplinary Brain and Behavioral Sciences, Crean College of Health and Behavioral Sciences, Schmid College of Science and Technology, Chapman University, Orange, CA, USA', 'includeInCitation': False, 'sc
hemaKey': 'Organization'}, {'name': 'Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleNa
me': ['dandi:Author', 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'suneil.kalia@uhn.ca', 'name': 'Kalia, Suneil K.', 'affiliation': [{'name': 'Division of Neurosurgery, Department of Surgery, University of Toronto, Toronto, Canada', 'includeInCitatio
n': False, 'schemaKey': 'Organization'}, {'name': 'Krembil Brain Institute, Toronto Western Hospital, Toronto, Canada', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author',
 'dandi:ProjectMember', 'dandi:Validation'], 'email': 'Taufik.Valiante@uhn.ca', 'name': 'Valiante, Taufik A.', 'affiliation': [{'name': 'Krembil Brain Institute, Toronto Western Hospital, Toronto, Canada', 'includeInCitation': False, 'schemaKey': 'Organizatio
n'}, {'name': 'Division of Neurosurgery, Department of Surgery, University of Toronto, Toronto, Canada', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:Project
Member', 'dandi:Validation'], 'email': 'Adam.Mamelak@cshs.org', 'name': 'Mamelak, Adam N.', 'affiliation': [{'name': 'Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'i
ncludeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Author', 'dandi:Conceptualization', 'dandi:FundingAcquisition', 'dandi:ProjectMember', 'dandi:Resources', 'dandi:Software', 'dandi:Supervision', 'dandi:Validation'], 'identifier': '0000-000
2-9207-7069', 'email': 'Ueli.Rutishauser@cshs.org', 'name': 'Rutishauser, Ueli', 'affiliation': [{'name': 'Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'name': 'Depa
rtment of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'name': 'Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA', 'includeInCita
tion': False, 'schemaKey': 'Organization'}, {'name': 'Computational and Neural Systems Program, California Institute of Technology, Pasadena, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'name': 'Center for Neural Science and Medicine,
 Department of Biomedical Science, Cedars-Sinai Medical Center, Los Angeles, CA, USA', 'includeInCitation': False, 'schemaKey': 'Organization'}], 'includeInCitation': True, 'schemaKey': 'Person'}, {'roleName': ['dandi:Sponsor'], 'awardNumber': 'U01NS103792', 
'name': 'Stroke, National Institute of Neurological Disorders and', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'roleName': ['dandi:Sponsor'], 'awardNumber': '1554105', 'name': 'Foundation, National Science', 'includeInCitation': False, 'schema
Key': 'Organization'}, {'roleName': ['dandi:Sponsor'], 'awardNumber': 'R01MH110831', 'name': 'Health, National Institute of Mental', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'roleName': ['dandi:Sponsor'], 'name': 'Neuroscience, McKnight Endo
wment for', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'roleName': ['dandi:Sponsor'], 'name': 'Foundation, NARSAD Young Investigator grant from the Brain & Behavior Research', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'roleNam
e': ['dandi:Sponsor'], 'name': 'Foundation, Kavli', 'includeInCitation': False, 'schemaKey': 'Organization'}, {'roleName': ['dandi:Sponsor'], 'awardNumber': 'U19NS104590', 'name': 'initiative, BRAIN', 'includeInCitation': False, 'schemaKey': 'Organization'}],
 'description': 'A challenge for data sharing in systems neuroscience is the multitude of different data formats used. Neurodata Without Borders: Neurophysiology 2.0 (NWB:N) has emerged as a standardized data format for the storage of cellular-level data toge
ther with meta-data, stimulus information, and behavior. A key next step to facilitate NWB:N adoption is to provide easy to use processing pipelines to import/export data from/to NWB:N. Here, we present a NWB-formatted dataset of 1863 single neurons recorded 
from the medial temporal lobes of 59 human subjects undergoing intracranial monitoring while they performed a recognition memory task. We provide code to analyze and export/import stimuli, behavior, and electrophysiological recordings to/from NWB in both MATL
AB and Python. The data files are NWB:N compliant, which affords interoperability between programming languages and operating systems. This combined data and code release is a case study for how to utilize NWB:N for human single-neuron recordings and enables 
easy re-use of this hard-to-obtain data for both teaching and research on the mechanisms of human memory.', 'identifier': 'DANDI:000004', 'keywords': ['cognitive neuroscience', 'data standardization', 'decision making', 'declarative memory', 'neurophysiology'
, 'neurosurgery', 'NWB', 'open source', 'single-neurons'], 'license': ['dandi:CCBY40'], 'name': 'A NWB-based dataset and processing pipeline of human single-neuron activity during a declarative memory task', 'schemaVersion': '1.0.0-rc1', 'repository': 'https:
//dandiarchive.org/'}}

more human friendly metadata record is likely to be up to date in https://github.com/dandi/dandi-api-datasets/blob/master/000004/dandiset.yaml ;)

yarikoptic commented 3 years ago

oh, I thought it might be a duplicate of https://github.com/dandi/dandi-api/issues/88 but that one is now addressed by https://github.com/dandi/dandi-api/pull/93 (merged 2 hours ago) but I still get 500 (may be API server instance is not yet updated to reflect)

waxlamp commented 3 years ago

@yarikoptic, are you still getting a 500 with this dandiset?

yarikoptic commented 3 years ago

it created fine now : https://gui-beta-dandiarchive-org.netlify.app/#/dandiset/000004 thanks for the reminder to check, closing