openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

How can we view training data results? #620

Closed fathollahzadeh closed 1 month ago

fathollahzadeh commented 1 month ago

Hello, Based on my current understanding, the system only reports test results. Is there an option to view the training data results as well?

PGijsbers commented 1 month ago

This might not be exactly what you are looking for, but some frameworks support a _save_artifacts parameter from the framework configuration. See here an example for autogluon and h2o. Whether a framework supports this depends on their integration, and which items can be saved also depends on the framework. So some inspection of the framework's exec.py is necessary. Typically you will find something like a save_artifacts function if they do support it.