support python runtimes for different versions

[jdries]

openEO backend wants to be able to upgrade python versions, while offering a migration path for UDF's. The best way forward seems to add support for running batch jobs in a specific python runtime, as indicated in the UDF. The assumption here is that process graphs with multiple UDF's all use the same runtime.

We can then run a batch job with a specific python version, allowing users to gradually upgrade their UDF's.

[jdries]

Implementation note: there effectively exists an official python3.11 package for almalinux8: https://repo.almalinux.org/almalinux/8/AppStream/x86_64/os/ https://almalinux.pkgs.org/8/almalinux-appstream-x86_64/python3.11-3.11.5-1.el8_9.x86_64.rpm.html

Meaning it's effectively an option to provide a python3.11 install next to our current default python3.8. I would suggest to keep it low on extra packages, if we're anyway going to try and use more dynamic package installation.

[jdries]

This is already quite a bit further now: we have a working python 3.11 image Main things to do is remove various hardcoded 'python3.8' references in openEO code. Environment variables in the docker images themselves should point to the right python to use, and configure things like python path.

[soxofaan]

Main things to do is remove various hardcoded 'python3.8' references in openEO code

:+1: yes I also noticed we have way to much of these hardcoded "python3.8" references in various places

[jdries]

I now added an image-name job option, allowing to override it if you know the image.

For a better solution, we somewhere need to map a fixed runtime name like 'python 3.11' to the corresponding image name. Maybe this can be put in a configmap or zookeeper.

[jdries]

Moving this one back into the backlog: needs some devops work to map container images to runtime names.