search

City-of-Helsinki / mlops-template

Generic repository template for small scale MLOps
Apache License 2.0
24 stars 8 forks source link

add python versions 3.9, 3.10 and 3.11 #27

Closed nakytoe closed 1 year ago

nakytoe commented 2 years ago

Currently the template is built on python 3.8. The user can manually change the python version, but this requires lot of manual work. We should at least provide python versions 3.9 and 3.10

Figure out how to do this so that usage is as easy as possible (one template or template per python version)

nakytoe commented 2 years ago

Not critical for Helsinki use, but nice to have

jussiarpalahti commented 2 years ago

@nakytoe Could you make a check list of things that should be changed? Beyond settings.ini minimum version and Dockerfile I didn't find places that should be changed. I'd surmise bigger problems might come from dependency mismatches.

nakytoe commented 2 years ago

1) settings.ini & Dockerfile 2) dependencies of core packages (I tried once converting the template to 3.9 and there were some issues with dependencies) 3) instructions 4) how to distribute template for different python versions as easily as possible. Changing settings in multiple files & solving dependency issues is just too much

nakytoe commented 2 years ago

However, I think this is something we don't have to solve immediately, because the py version does not really matter for our current projects. So perhaps in v0.0.3 or later. Of course, if the solution is super easy, we can fix it asap, but I would not put hours to it for now

jussiarpalahti commented 2 years ago

Python 3.11 is coming with supposedly big speed gains: https://mobile.twitter.com/methane/status/1501108529297432577 And 3.10 was already many ways faster. So, there's that. Granted, 3.11 is far away now.

I'm thinking template could be set to highest possible Python version. Dependencies tend to get updated (I'd wager) when template based repo is first used. Adding project's dependencies very likely leads to updates to base deps also. If base Dockerfile can support building container for chosen Python version then there shouldn't be anything else to set. Though creating this version matrix for containerization might not be simple.

nakytoe commented 2 years ago

Good point. Let's add 3.11 already to this. I suppose it is easy to update versions after we figure out how to do it with a couple of them

nakytoe commented 1 year ago

possible by setting the container to use virtual environment