GeoscienceAustralia / dea-sandbox

Digital Earth Australia Sandbox config and planning
Apache License 2.0
13 stars 6 forks source link

Broken environment due to numpy version #219

Closed Kirill888 closed 1 year ago

Kirill888 commented 2 years ago

Describe the bug

python environment has broken prackages, for example numba doesn't import since it requires older version of numpy than what is installed. I have reported it here before: https://github.com/digitalearthafrica/deafrica-sandbox/issues/1 but looks like image is built in this repo.

Run pip check on sandbox to see what libs are affected, I care about numba but other ones are also not happy.

Reason for failure is probably this section of Dockerfile:

https://github.com/GeoscienceAustralia/dea-sandbox/blob/c86423bc8eb1f587ea072ed1b2a0693856e16b49/docker/Dockerfile#L121-L126

Several things are wrong with that

  1. Running as root while environment is owned by jovyan user, so this installs packages that one can't patch at runtime on the sandbox as they are owned by root
  2. Updating all dependencies of hdstats without taking into consideration constraints supplied previously, as a result newer version of numpy is pulled, in breaking numba and libraries that depend on that.

I suggest to at least run as jovyan and not root and use --no-deps flag and to ensure only hdstats is updated and not it's dependencies, or better yet make sure to install correct version from the get go.