search

SainsburyWellcomeCentre / aeon_mecha

Project Aeon's main library for interfacing with acquired data. Contains modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines.
BSD 3-Clause "New" or "Revised" License
6 stars 6 forks source link

Pose reader assumes all models are trained on the same HPC node #434

Closed anayapouget closed 1 month ago

anayapouget commented 1 month ago

With the recent changes to the pose reader, it's now necessary to include the name of the HPC node on which the model was trained as part of the reader pattern in order for it to be able to find the config file . E.g., I used to use aeon.io.reader.Pose(pattern="CameraTop_202") and now I need to use aeon.io.reader.Pose(pattern="CameraTop202\<HPC-node-name>") .

My understanding is that in theory, the reader pattern should be an indication of the type of data you want to read right? So I feel that including the HPC node name is a bit strange.

But past that, it also introduces some issues. For example, say that midway through an experiment the sleap identity model is noticed to not be performing great and is switched out for a new one that was trained on a different HPC node. Now some of the sleap data is named "CameraTop_202_test-node1..." and some is named "CameraTop_202_node-0...". If you are interested in loading and viewing the data, you would have to call aeon.load twice due to the difference in HPC node name, despite the fact that the data you are trying to read is of exactly the same type.

Do we maybe want to change the pose reader again so that we can go back to using CameraTop_202 without including the HPC node name?

anayapouget commented 1 month ago

Nevermind I got confused but actually it's ok if you specify the reader as aeon.io.reader.Pose(pattern="CameraTop202*"). You just need the extra underscore at the end.