search

databricks / mlops-stacks

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.
https://docs.databricks.com/en/dev-tools/bundles/mlops-stacks.html
Apache License 2.0
416 stars 141 forks source link

Update the version of matplotlib to avoid security issues #155

Closed lbcommer closed 4 months ago

lbcommer commented 4 months ago

Update the version of matplotlib to avoid security issues with the version of its required package Pillow. This way, the installed version of Pillow will be at least 10.3.0, which doesn't have the problematic security issue: https://security.snyk.io/vuln/SNYK-PYTHON-PILLOW-5918878

arpitjasa-db commented 4 months ago

Thanks for the PR @lbcommer! It seems though that we're running into some version dependency and incompatibility issues with a naive upgrade. This can be something we look into and update all our dependencies accordingly

lbcommer commented 4 months ago

@arpitjasa-db This security vulnerability is scored as critical. Any new project created with the template will introduce the security risk. I think it makes sense to apply a quick solution. That's not against any other further improvement/update of the rest of the dependencies.

arpitjasa-db commented 4 months ago

@lbcommer yeah I understand, but what I meant was this upgrade seems to break MLOps Stacks due to incompatible dependencies not resolving correctly, as we see from the above checks that ran.

lbcommer commented 4 months ago

@arpitjasa-db I see the problem. I modified the PR to include in the requirements the pillow package instead, since a version without the security issue (10.0.1). This way, all tests have passed. The last matplotlib works with pillow>=8 (https://github.com/matplotlib/matplotlib/blob/cbdd3b373ecf6074a8813465c16721d889852642/environment.yml#L22) and matplotllib 3.5.2 requires pillow>=6.2 so, all should be ok.

arpitjasa-db commented 4 months ago

@lbcommer fantastic thanks a bunch for making this change!