search

AllenNeuralDynamics / dynamic-foraging-task

Bonsai/Harp workflow for Dynamic Foraging with Python GUI for visualization and control
MIT License
5 stars 4 forks source link

Generating metadata for old sessions #482

Open XX-Yin opened 1 month ago

XX-Yin commented 1 month ago

We need to generate session and rig metadata for old sessions with some fields missing.

hagikent commented 1 week ago

I have been working on this as a part (and extension) of photometry hackathon. We have roughly three data categories:

  1. 20240528-; basically have rig and session metadata (session meta generation success rate is low, ~80%)
  2. 2024 early (-0527); rig metadata backfilling is done; session metadata backfilling will be done next week
  3. 2023 late; rig names are different (eg Blue-1), I will finish this in 2-3days; session metadata backfilling will be done next week

adding @rachelstephlee for potential help

hagikent commented 2 days ago

2 and 3 are done and ready for mass-uploading. I would wait for a while for 1; session metadata generation success rate issue (#569 ) should be solved first.

hagikent commented 2 days ago

@alexpiet @XX-Yin @hanhou @ZhixiaoSu @galenlynch

session.json and rig.json metadata backfilling is done for old data and organized into the following locations. Before mass-uploading, it would be great if you could take a look. I will start uploading the data early next week (7/15-). Note, some info is inevitably inaccurate – e.g. as rig config has been frequently changed and impossible to track changes, rig.json is defined in the future relative to when data collection was done.

  1. 2024 early (-0527): \allen\aind\scratch\svc_aind_behavior_transfer\2024early_DataNoMetadata_Reorganized (1432session, 69sessions don't have metadata due to either no-behavior or stopped immediately)

  2. 2023 late: \allen\aind\svc_aind_behavior_transfer\2023late_DataNoMeta_Reorganized (179sessions, 19sessions don't have metadata)