search

TeamEpochGithub / agogos

Package that creates the underlying construction of a machine learning pipeline [DEPRECATED]
MIT License
4 stars 0 forks source link

ENH: Add callbacks in the Training/Transforming Systems #70

Closed Jeffrey-Lim closed 1 month ago

Jeffrey-Lim commented 6 months ago

The Training and Transforming Systems currently run a for loop that you have no control over. We should add a callback system (similar to Lightning Callbacks) that allows you to run any function at the start and end of each step.

One obvious use case for this would be to get a TQDM bar for a TransformationPipeline in Epochalyst.

schobbejak commented 6 months ago

You can add a tqdm per block, why would you want a tqdm bar for the whole pipeline?

Jeffrey-Lim commented 6 months ago

You can add a tqdm per block, why would you want a tqdm bar for the whole pipeline?

For some blocks you can't add a reliable tqdm bar. Additionally, you can have a pipeline with multiple steps that each complete somewhat quickly and it would be more useful to see a progress bar for the whole pipeline instead.

A tqdm bars is just one use case though. You might want to do some special logging or caching.