fedora s2i python

[juliusvonkohout]

Hello, i would just like to ask whether you could add support for the fedora python3 s2i image.

https://registry.fedoraproject.org/repo/f31/python3/tags/ podman pull registry.fedoraproject.org/f31/python3:0

I built the minimal image and just got the warning

/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
...
+ cd /opt/app-root/gateway
+ source scl_source enable rh-nodejs10
/tmp/scripts/assemble: line 95: scl_source: No such file or directory
error: build error: non-zero (13) exit code from registry.fedoraproject.org/f31/python3@sha256:31f5c4a93c85b88621b6becb032f500f645645f8e5b6cae8592e87f03612e79e

We need fedora because there wont be a centos8 s2i python image as declared here https://github.com/sclorg/s2i-python-container/pull/347#issuecomment-555367696

You mention scl_enable support here https://github.com/sclorg/s2i-python-container/pull/328 and they seem to be waiting on your feedback https://github.com/sclorg/s2i-python-container/pull/328#issuecomment-572063821

[GrahamDumpleton]

What is the underlying requirement for having this be able to work with the Fedora 32 S2I image that isn't satisfied by the existing CentOS based S2I images?

I am not involved in any way with the Red Hat Python S2I images these days and thus why I haven't commented. Lack of scl_enable is an issue for the S2I project developers to work out and decide on whether they are going to provide compatibility with existing usage or break compatibility.

[juliusvonkohout]

Centos7 is getting quite old. For example texlive (and several other) packages become incompatible with some python packages or are missing features. Furthermore perfomance wise a new compiler toolchain is desireable. And in addition it is python 3.7 in the fedora image.

rhel8ubi is not usable, because you cannot install e.g R-core as verfied by the RedHat developers in https://github.com/sclorg/s2i-python-container/pull/347#issuecomment-555367696

So should it work if they add scl_enable as proposed in https://github.com/sclorg/s2i-python-container/pull/328 , or are there further limitations that you know?

[GrahamDumpleton]

Agree that UBI images aren't really usable for serious development. Can't see that changing.

Do your images even need to be RHEL/CentOS/Fedora based? What about Debian based images?

In other words, would using something based off the official Jupyter project base images work for you. That does mean Anaconda Python though, but then, that is defacto Python for Python data science work anyway and many packages are only available for it which aren't available for PSF Python.

There seems to be a lack of any commitment from Red Hat in supporting anything newer than Python 3.6 so may be more sensible to swap to the Jupyter project base images, layering on top the extra smarts that this projects images had.

[juliusvonkohout]

Well the problem is that you use s2i to build and I use s2i in further build steps too. Changing that might be a lot of effort. Furthermore i definitely prefer Fedora to Debian and Ubuntu. I also do not see the need for centos/rhel instead of fedora since this is a developer workstation.

The fedora S2I image uses Python 3.7

[julius@Julius-ASUS ~]$ podman run registry.fedoraproject.org/f31/python3:0 python3 --version
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Python 3.7.4

I think 3.7 will be fine for a long time given the python release cadence.

3.6 | 3.6.10 | 2016-12-23
3.7 | 3.7.6 | 2018-06-27
3.8 | 3.8.1 | 2019-10-14

A RHEL8 subscription might be an option, but i would prefer to get rid of the use of scl_enable. Without scl_enable I could directly use the recent fedora python s2i images. Is this something you would consider? E.g. using bashrc or profile instead of scl_enable if applicable ?

[GrahamDumpleton]

I haven't relied on S2I builds to create the actual base notebook image for quite a while and use a Dockerfile build instead. Even if use Jupyter project base image instead, one can still configure it to be S2I enabled. You can see an example of that at:

• https://github.com/jupyter/docker-stacks/tree/master/examples/source-to-image

The problem with using the Jupyter project base images is how conda handles virtual environments and how that interacts with trying to back the Jupyter notebook workspace with a persistent volume. I sort of worked out a way of handling that a long time ago, but it was problematic due to bugs in conda handling of virtual environments. I will have to work it all out again as forgotten what I did.

In the long run I would prefer to standardise on using the Jupyter project base images because that is what most want to use, and there is so many unknowns around roadmap for S2I base images for Python from Red Hat and Fedora projects. Python isn't a focus of Red Hat as they only really care about Java and Node.js, so their support for Python is always going to be sub par, which is really bad for people wanting to do datascience with Python as various stuff isn't going to be able to work.

[juliusvonkohout]

Alright, i was able to build what i need with jupyter/r-notebook as pure docker build. I guess jupyter/minimal-notebook should also be fine.

"The problem with using the Jupyter project base images is how conda handles virtual environments and how that interacts with trying to back the Jupyter notebook workspace with a persistent volume."

The persistent volume support is crucial for me. Maybe it is already fixed in the currently used Conda 4.7.

If you manage to use jupyter/minimal-notebook as base, you will save a lot of maintenance in the long run. Anyone who wants more can just use the larger images for a build or do it himself.

I wish you good luck with the changes and can test if necessary. Do you plan to do it in the near future?

I guess jupyterhub can stay as it is, or do you plan to use https://hub.docker.com/r/jupyterhub/jupyterhub?

[GrahamDumpleton]

Nothing can happen until February. I am overseas and have limited time, but also have rather poor Wifi in the hotel, so pulling down the very large Jupyter project images to work on them is not going to be possible.

Ping me again start of February to make sure I haven't forgotten.

[GrahamDumpleton]

And no I would still use the JupyterHub image have now as it is set up to do a lot more, and easier, than the official one.

[juliusvonkohout]

Nothing can happen until February. I am overseas and have limited time, but also have rather poor Wifi in the hotel, so pulling down the very large Jupyter project images to work on them is not going to be possible.

Ping me again start of February to make sure I haven't forgotten.

i would also like to ask whether you could upgrade to jupyterlab 2.1.x

[juliusvonkohout]

@GrahamDumpleton are you not interested in the project anymore?

[GrahamDumpleton]

I did leave Red Hat and don't work specifically with OpenShift anymore, so my priorities are definitely elsewhere. I have also been super busy on another project as well. No one connected with Red Hat has ever asked about taking over the project. As I understand it Red Hat people forked/copied it all for Open Data Hub and use their own variations so don't seem to care about the original project.

Is all you are after at this point the newer Jupyter Lab version? Since you have added the comment about the newer Jupyter Lab version on an issue which talks about a lot of other stuff, rather than creating a separate issue clearly stating want just that, is a bit confusing what you want.

[juliusvonkohout]

It is mostly about a newer base image with s2i support. E.g. the fedora support. We are just updating jupyterlab in another s2i step currently. It would be enough if you just finish your discussion https://github.com/sclorg/s2i-python-container/pull/328#issuecomment-572063821 and scl_enable is merged. Then i should be able to just use fedora images.