Usually (in training and evaluation) we do this using MLflow.
However, it may be more complex to integrate MLflow in the inference step than in the other cases. I think it may require to define a dataloader from a video (see #238).
In the meantime, it may be a good idea to implement a simple way of keeping track of basic metrics:
detector model used,
tracking config used,
in the bash script for an array job in the HPC we save the tracking config along the output data (implemented in #251 , but we don't do this when we run the command locally. A "local solution" that extends to the bash script would be ideal.
ground truth used if available,
selected outputs (are videos saved, are frames saved),
Usually (in training and evaluation) we do this using MLflow.
However, it may be more complex to integrate MLflow in the inference step than in the other cases. I think it may require to define a dataloader from a video (see #238).
In the meantime, it may be a good idea to implement a simple way of keeping track of basic metrics: