Implement monitoring possibility for AI models with existing tool

[PhilippDrechsler]

Running models during training and in production must have the possibility to be monitored. several tools are available. Please consider to implement one of the following tools (open source) :

• mlflow (https://mlflow.org/)
• airflow (https://airflow.apache.org/)

More tools are listed here:

• https://hauke.me/speaking/tools-for-mlops/
• https://hauke.me/speaking/tools-for-monitoring-ml-systems/

[jonau]

Could you please emphasize on what monitoring functionality you need?

In general the architecture as a general purpose (i.e. framework independent) architecture does not allow deep monitoring of machine learning models (comparable to e.g. "weights & bias").

On the other hand monitoring of production and deployment related metrics is "in scope" of this project.