search

AllenNeuralDynamics / aind-data-schema

A library that defines AIND data schema and validates JSON.
MIT License
23 stars 17 forks source link

Add code_url, code_version to QCMetric to ensure reproducibility #1114

Open seanmcculloch opened 1 week ago

seanmcculloch commented 1 week ago

Is your feature request related to a problem? Please describe. The calculation of metric values, images, etc is currently not tracked in the QualityControl object.

Describe the solution you'd like I would like code_url and code_version to be added to QCMetric (similar to processing.json) so that code relevant to the metric can get tracked.

Describe alternatives you've considered Considered adding this information to the QualityControl or QCEvaluation classes, but adding it to QCMetric is more intuitive and allows for more detailed tracking.

dbirman commented 1 week ago

Thanks for bringing this up @seanmcculloch. Is adding code_url/code_version here a duplication of tracking that's already being in the processing.json file?

We definitely want this information, but I think it would make sense to add the QC-generating processes into processing and track this information there. Then the question is how to link metrics to the process that generated them. Maybe we put the name of the metric with a prefix, like qcmetric-__name__ in the outputs field https://github.com/AllenNeuralDynamics/aind-data-schema/blob/dbb4d994995089867c507e3d3045af0e5b5c912c/src/aind_data_schema/core/processing.py#L68C1-L68C102

From a developer standpoint it will be annoying if you want to quickly go from a metric back to find it's process, but we can add a helper function to simplify that.