search

mlflow / mlflow-example

An example MLflow project
Apache License 2.0
237 stars 333 forks source link

How to upload an artifact only for selected run_id in MLFlow ? #6

Open vackysh opened 4 years ago

vackysh commented 4 years ago

Hi,

I am aware that we can set default artifact root option to S3 bucket in mlflow server

mlflow server \ --backend-store-uri /mnt/persistent-disk \ --default-artifact-root s3://my-mlflow-bucket/ \ --host 0.0.0.0

By doing this, we are setting default storage for artifacts to S3 buckets for all runs. As a data scientist, I would be training model multiple times and then comparing using MLFlow Tracking UI. I would always suggest to store artifacts at local file system and only push best model to S3 bucket. Basically I am implementing CI/CD pipeline for MLFlow -sagemaker. As soon as best model pushed to S3 bucket, i would trigger CI/CD pipeline and deploy the model to AWS sagemaker endpoint and then using API Gateway i would create the REST API.

Do we have any such MLFlow API which can be used to push the model only for particular run_id ? From documentation, i couldn't find any such API.

Regards, Vackysh

vackysh commented 4 years ago

Hi @Zangr ,

Could i expect some reply for my query ?

Regards, Varun

Borda commented 4 years ago

You may consider use own MlflowClient see https://mlflow.org/docs/latest/python_api/mlflow.tracking.html#mlflow.tracking.MlflowClient