search

dandisets / 000053

359 files, 1.4 TB, 1 release, Recordings from medial entorhinal cortex during linear track and open exploration
https://identifiers.org/DANDI:000053
0 stars 0 forks source link

girder: empty items #1

Open yarikoptic opened 3 years ago

yarikoptic commented 3 years ago

DANDI:000053 Empty item with prefix=sub-npI1/: {'_id': '5fe272551eee1c6740c2d636', '_modelType': 'item', 'baseParentId': '5e59bb0af19e820ab6ea6c62', 'baseParentType': 'collection', 'created': '2020-12-22T22:25:25.534000+00:00', 'creatorId': '5da4b8fe51c340795cb18fd0', 'description': '', 'folderId': '5fe272551eee1c6740c2d635', 'meta': {'experiment_description': 'trial contrast: 100', 'identifier': '0413_baseline_2', 'institution': 'stanford university', 'lab': 'giocomo', 'md5': '136d964a399f536cb9b828258b8c8121', 'nwb_version': '2.2.5', 'session_description': 'straight track.', 'session_start_time': '2019-04-13T00:00:00-07:00', 'sha1': 'f87b3fcaec1775b8ceb971eb2fec1c429b5613bc', 'sha256': 'e577e3e995cffbe8b58e56b360c3f424394093147461e99b9ca811fcc12a1c78', 'sha512': 'e5620608f27c31484dcabd035e12cd386f2e7ec7ec63df44784b6d749e71ebf4f1e62b46237714577ec13e77f9b8cc711e25a0be6ce9c4ceae5fb1e48be9d760', 'subject_id': 'npi1', 'uploaded_by': 'dandi 0.10.0', 'uploaded_datetime': '2020-12-22T17:29:17.680182-05:00', 'uploaded_mtime': '2020-12-22T15:20:04.650000-05:00', 'uploaded_nwb_object_id': '325ce9d6-2dd2-4d43-933b-77fb439893bd', 'uploaded_size': 19000808}, 'name': 'sub-npI1_ses-20190413_behavior.nwb', 'size': 19000808, 'updated': '2020-12-22T22:29:38.842000+00:00'} 359 assets the same

ref: https://github.com/dandi/dandi-cli/pull/589

satra commented 3 years ago

@CodyCBakerPhD - here is one empty item we found when we migrated. could you please update this item and then let us know here. we can close this issue.

CodyCBakerPhD commented 3 years ago

What's the empty item?

satra commented 3 years ago

sub-npI1/sub-npI1_ses-20190413_behavior.nwb

the original post contains more details on the item.

CodyCBakerPhD commented 3 years ago

Hm... so the entire original data file never finished upload? I'll see if I'm able to reconvert and upload

satra commented 3 years ago

correct - this should not happen with the new CLI, but in the older one, this was a possibility.

CodyCBakerPhD commented 3 years ago

I re-converted the data and did a dandi upload for the first time since the new release. Strangely, the automatically chosen name of the file did not fill in 'ses-20190413' in between 'sub-I1' and 'behavior+ecephys'. I had to manually alter that after the dandi organize step in order for it to show up as it originally did.

Does the problem now appear corrected on your end?

yarikoptic commented 3 years ago

I believe it should've not happened. You uploaded this one already? Did you point organize to just that file while others were already in place?

CodyCBakerPhD commented 3 years ago

The workflow was as follows:

1) Re-downloaded that one single session of original data (the entire dataset had been removed after previous upload) 2) Re-converted that single session to NWB 3) Moved single NWB file to a folder 4) ran validate (no errors) on the folder 5) Downloaded with existing dandiset with the --download dandiset.yaml option (since I don't need to re-download the entire dandiset) 6) from within the dandiset folder, called dandi organize ../nwb_folder -f dry followed by the symlink. This created the folder structure of sub_I1 as usual, but the file within it did not have the extra ses-### bit

From there, I manually altered the name of the file to be appropriate and proceeded with dandi upload. Does the girder thing now show up as non-empty?

satra commented 3 years ago

@yarikoptic - has the session extraction issue been resolved here?

@CodyCBakerPhD - girder is no longer in play. everything now goes through our API. you can check information on the server using dandi ls -r or using the API (see https://github.com/dandisets/000026/issues/1#issuecomment-828489913 for an example).