Open NickleDave opened 1 month ago
Just realized lightning has an mlflow logger https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.loggers.mlflow.html#mlflow-logger so we can perhaps somewhat painlessly integrate that way -- pass in experiment name, run name, etc
edit: looking at this again and related issues, questions in the Discussions, etc., my impression is that integration is not perfect and it's probably less painful in the long run to keep them separated if possible
edit edit: basically I need to implement the bad version of this and see if it can even work
pip install vak[expt-tracking]
should add this optional dependency and then allow experiment trackingI don't want to invest a lot right now in supporting every feature of mlflow but it's clear to me we need some sort of experiment tracking (as @achabotl wisely pointed out many moons ago).
Prefer going with mlflow for now since it's the one I'm most familiar with and I would rather just adopt something that's widely used in industry, although sth more lightweight and application-agnostic like sacred or sumatra might be worth thinking about