search

mlflow / mlflow-export-import

Apache License 2.0
132 stars 78 forks source link

Implement bulk import all #7

Closed amesar closed 2 years ago

amesar commented 2 years ago

Implement

Related to: https://github.com/mlflow/mlflow-export-import/issues/11 - Make export_all work correctly

amesar commented 2 years ago

Import all objects is complex since we have to link the new registered model's version run ID with the already imported experiments and runs. The main problem is scale. If the number of runs can fit into memory, we can use an in-memory hash table of source and destination runs. If not, then what to do? A key value store?

amesar commented 2 years ago

Implemented as a console script import-all which calls import-models since the export directory structure is the same for both.