[bitnami/mlflow] Request -> support for MySQL in MLFlow

surya9teja commented 7 months ago

Name and Version

bitnami/mlflow 2.10.0

What is the problem this feature will solve?

Current MLFlow Helm chart versions lack native support for MySQL as a backend database, limiting flexibility and integration with existing infrastructure.
Users with MySQL databases in their Kubernetes clusters cannot seamlessly leverage them for MLFlow tracking and experiment management.

This forces users to either:

Introduce additional database technologies (e.g., PostgreSQL) solely for MLFlow, increasing complexity and potential overhead.
Forgo the benefits of a centralized database, hindering collaboration and data management.

What is the feature you are proposing to solve the problem?

Proposed Feature:

Integrate MySQL support into the bitnami/mlflow Helm chart:
Enable configuration of MySQL database connection details through Helm chart values.
Implement necessary changes within the chart to enable MLFlow to utilize MySQL for tracking and metadata storage.

Benefits:

Unified Database Environment: Streamline ML workflows and data management by aligning MLFlow with existing MySQL-based systems.
Enhanced Flexibility and Integration: Allow users to leverage their preferred database technology within Kubernetes clusters, reducing operational overhead and complexity.
Cost Optimization: Potentially reduce costs by eliminating the need to provision and manage separate databases for MLFlow.
Widespread Adoption: MySQL's popularity in the machine learning community ensures a large potential user base for this feature.

What alternatives have you considered?

I was considering helm chart from community charts which supports MySQL but the appversion is old 2.1.1 vs bitanami's charts app version 2.10.0 (which is latest).

javsalgar commented 7 months ago

Hi!

Thank you so much for the feature request. I will forward this to the team for evaluation.

For the time being, it is possible for you to integrate it with an existing mysql installation by setting postgresql.enabled=false and overriding the command and args sections (see command and args under tracking) to set the configuration to use your own mysql database.

surya9teja commented 7 months ago

Thanks for the suggestion and I will use the custom command to setup MySQL for now.

act-mreeves commented 5 months ago

I tried this and got pretty far by hacking on the helm chart. For example the helpers are very opinionated about postgres and I had to modify the URIs to be able to change the driver/protocol: https://github.com/bitnami/charts/blob/main/bitnami/mlflow/templates/_helpers.tpl#L495

- {{- printf "postgresql://%s:$(MLFLOW_DATABASE_PASSWORD)@%s:%v/%s" (include "mlflow.v0.database.user" .) (include "mlflow.v0.database.host" .) (include "mlflow.v0.database.port" .) (include "mlflow.v0.database-auth.name" .) -}}
+ {{- printf "%s://%s:$(MLFLOW_DATABASE_PASSWORD)@%s:%v/%s" (include "mlflow.v0.database.protocol" .) (include "mlflow.v0.database.user" .) (include "mlflow.v0.database.host" .) (include "mlflow.v0.database.port" .) (include "mlflow.v0.database-auth.name" .) -}}

However, the base image for mlflow would need to have additional system and/or pip dependencies installed to support this.

For example: I set protocol to mysql+pymysql trying to deal with an error similar to ImportError: No module named MySQLdb. https://stackoverflow.com/questions/22252397/importerror-no-module-named-mysqldb. Any chance of there being an image we can use with those dependencies installed? I'd be happy to make a PR if so. https://bitnami.com/stack/mlflow/containers says I'd probably need to add that here? a PR to here https://github.com/bitnami/containers/blob/main/bitnami/mlflow/2/debian-12/Dockerfile ?

javsalgar commented 5 months ago

Hi!

I forwarded this to the team to see if we can add the module to the container.

act-mreeves commented 5 months ago

btw @javsalgar I wanted to mention this mlflow community chart does support mysql. See https://github.com/burakince/mlflow/blob/main/Dockerfile and it's built off of this: https://github.com/docker-library/python/blob/1b7a1106674a21e699b155cbd53bf39387284cca/3.9/slim-bookworm/Dockerfile. Not clear to me how they support https://github.com/community-charts/helm-charts/blob/main/charts/mlflow/values.yaml#L102C1-L103C22.

    # -- mysql database connection driver. e.g.: "pymysql"
    driver: "pymysql"

Thanks!

amybachir commented 3 months ago

btw @javsalgar I wanted to mention this mlflow community chart does support mysql. See https://github.com/burakince/mlflow/blob/main/Dockerfile and it's built off of this: https://github.com/docker-library/python/blob/1b7a1106674a21e699b155cbd53bf39387284cca/3.9/slim-bookworm/Dockerfile. Not clear to me how they support https://github.com/community-charts/helm-charts/blob/main/charts/mlflow/values.yaml#L102C1-L103C22.
    # -- mysql database connection driver. e.g.: "pymysql"
    driver: "pymysql"
Thanks!

yea but the chart has not been updated in over a year. It would be great if bitnami's mlflow chart adds native MySQL support

bitnami / charts