search

SainsburyWellcomeCentre / aeon_mecha

Project Aeon's main library for interfacing with acquired data. Contains modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines.
BSD 3-Clause "New" or "Revised" License
5 stars 6 forks source link

Load blob tracking data for animal position when missing SLEAP data #400

Open ttngu207 opened 2 months ago

ttngu207 commented 2 months ago

For parts of some experiments, e.g. the presocial periods, SLEAP data is not available. Thus, currently, animal position data is not available for those presocial periods.

However, the animal position blob tracking data is always available, just not ingested. It's worth to note that the blob tracking data cannot reliably identify the different animals when more than one animal present (thus we should use SLEAP data). But for single-animal time periods, this is a reasonable alternative.

The task here is to ingest blob tracking data whenever SLEAP data is not available, only for the periods of single-animal only

Tag @jkbhagatio and @OrsiFolsz here as well

anayapouget commented 1 month ago

We could also consider running full pose inference on pre and post social periods. This would be useful to Joaquin (he is currently just using some data in my personal ceph folders) and in general is would probably be good to have in order to be able to better compare individual and social sessions in analyses?

ttngu207 commented 1 month ago

Thanks @anayapouget , yes, that would certainly solve this issue. The question is whether we want to make this part of a formal process and to this in an automated manner.

  1. automatically triggers full pose analysis (send jobs to the clusters) on a per chunk (or per epoch?)
    • what should be the trigger, how to orchestrate that
    • how to configure the job(s) to send to the cluster correctly (iirc you need to do quite a bit of careful manual work to make this happen)
  2. once the jobs are finished, copy the results to the .../processed folder

If it's a one off fix, we can certainly use your help to do this once. But if we want to include this step formally in the larger data processing/analysis flow, we should have another discussion to engineer this.

jkbhagatio commented 1 month ago

We've decided to go with the approach of adding a check to use centroid blob data for position ingestion if no sleap data, else will use sleap data (this includes data that would be in the processed dir if full pose inference is run on single animals)