Project Aeon's main library for interfacing with acquired data. Contains modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines.
BSD 3-Clause "New" or "Revised" License
6
stars
6
forks
source link
Inconsistent finding of identities can crash Pose reader #420
Again it is unclear why this was not crashing in my remote CEPH environment, but there is a slight inconsistency in the way the method class_int2str extracts the class identities from the confmap_config.json file:
While these superficially seem functionally identical since classes is a sub-field of class_vectors, it crashes on a local JSON file in a local environment. It is all very mysterious, but may point to a couple of fundamental problems with the way we are approaching this nested key extraction.
For reference, making the former case equal to the latter solves the issue.
Another way to resolve the issue might be to dispense with class_int2str altogether. Since we are already retrieving the array of identities at an earlier step anyway, we can just work directly with that vector to convert the IDs into named labels, which would avoid having to scan the JSON file again.
While we are at it we might also as well resolve new FutureWarning messages in pandas raised at line 426:
FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value 'BAA-1104045' has dtype incompatible with float32, please explicitly cast to a compatible dtype first.
data.loc[data["identity"] == i, "identity"] = subj
Again it is unclear why this was not crashing in my remote CEPH environment, but there is a slight inconsistency in the way the method
class_int2strextracts the class identities from theconfmap_config.jsonfile:https://github.com/SainsburyWellcomeCentre/aeon_mecha/blob/7812b4fbbda6fd679fae26c926be2f09baaae781/aeon/io/reader.py#L417
For constrast, here is how we do it in the
get_class_namesmethod:https://github.com/SainsburyWellcomeCentre/aeon_mecha/blob/7812b4fbbda6fd679fae26c926be2f09baaae781/aeon/io/reader.py#L389
While these superficially seem functionally identical since
classesis a sub-field ofclass_vectors, it crashes on a local JSON file in a local environment. It is all very mysterious, but may point to a couple of fundamental problems with the way we are approaching this nested key extraction.For reference, making the former case equal to the latter solves the issue.
Another way to resolve the issue might be to dispense with
class_int2straltogether. Since we are already retrieving the array of identities at an earlier step anyway, we can just work directly with that vector to convert the IDs into named labels, which would avoid having to scan the JSON file again.While we are at it we might also as well resolve new
FutureWarningmessages inpandasraised at line 426:https://github.com/SainsburyWellcomeCentre/aeon_mecha/blob/7812b4fbbda6fd679fae26c926be2f09baaae781/aeon/io/reader.py#L421