Upgrade python version in Raspberry Pi image

[arist0v]

as a user i would like to be able to create an addon that require a specfic version of python(to meet library requirement) and have the gateway install it during the addon install process. (maybe using the manifest.json could make the gateway install the required python version in addition to the existing one but not as the default interpreter)

[benfrancis]

Which version of Python do you need? Perhaps we can ship an updated version with the 1.0.1 or 1.1 Raspberry Pi image. Unfortunately that would only apply to new installs though, since we can't automatically upgrade the underlying OS once installed (see #2801).

Bear in mind that people also run the WebThings Gateway software on lots of different operating systems through the .deb and .rpm packages, the docker image and even self-build. Having the gateway automatically install the requested version of Python when the add-on is installed would require platform specific implementation for each supported platform, and may be prone to errors. The add-ons system is not intended as a general purpose package manager for the OS.

One thing we could potentially do is add a member to the manifest member to define a minimum version of Python required by the add-on, like with the existing .strict_min_version and .strict_max_version members for gateway version. That way at least the add-on could only be displayed for gateways which are capable of running it, or potentially be displayed but with a warning that it requires a newer version of Python to run. That would allow the user to take action to manually install a newer version of Python if they want to.

[arist0v]

Since python 3,11 just get release, is suggest moving to 3.10 for now and wait a few time before going to 3.11

even if used on multiple OS, it should be possible to programmaticaly get the package manager (maybe using which to deterinate the package manager at least within the most popular one(apt yum etc.....) so we can programaticaly call it to add the required python version without overwriting the default one. something like for packageManager in list do which $packageManager if return /something/$packageManager then $packageManager install python3.XX

for the .deb or .rpm, it should be easier since you could add a config file or something specifiying it was install from a .deb or .rpm so it will be easier to call the relevant package manager.

i'm pretty sure i can code such behavior in python, so it could be possible in other language too(if you specie my the best language for it then i could try to do it in this language and you will just have to integrate the code where it will be the most appropriate.

talking here i figure i could probably add this feature in the main.py of my addon, but i think it could be a better feature to add it directly into the webthings code.

[benfrancis]

I've just realised that what I suggested is already implemented. When the gateway requests a list of add-ons it tells the server the system architecture, gateway version and node and python versions which are available.

e.g. https://api.webthings.io:8443/addons?arch=linux-arm&version=1.1.0&node=64&python=2.7,3.7

The manifest of add-ons can specify a required version of python or node, e.g. https://github.com/WebThingsIO/addon-list/blob/master/addons/date-time-adapter.json#L51

Only add-ons which are supported are displayed in the list.

even if used on multiple OS, it should be possible to programmaticaly get the package manager (maybe using which to deterinate the package manager at least within the most popular one(apt yum etc.....) so we can programaticaly call it to add the required python version without overwriting the default one.

Package management is very complicated. Doing this reliably in a cross-platform way would be extremely difficult. I'd suggest that system level package management is beyond the scope of what the gateway application should do.

i could probably add this feature in the main.py of my addon

FYI, I don't think your add-on would get approval to be added to the add-on list if it did this.

On reflection I think the current implementation is the right solution, so I'm going to close this issue.

[arist0v]

could you keep the issue to request the update to python 3.10 ???

Le ven. 28 oct. 2022, à 09 h 22, Ben Francis @.***> a écrit :

I've just realised that what I suggested is already implemented. When the gateway requests a list of add-ons it tells the server the system architecture, gateway version and node and python versions which are available.

e.g. https://api.webthings.io:8443/addons?arch=linux-arm&version=1.1.0&node=64&python=2.7,3.7

The manifest of add-ons can specify a required version of python or node, e.g. https://github.com/WebThingsIO/addon-list/blob/master/addons/date-time-adapter.json#L51

Only add-ons which are supported are displayed in the list.

even if used on multiple OS, it should be possible to programmaticaly get the package manager (maybe using which to deterinate the package manager at least within the most popular one(apt yum etc.....) so we can programaticaly call it to add the required python version without overwriting the default one.

Package management is very complicated. Doing this reliably in a cross-platform way would be extremely difficult. I'd suggest that system level package management is beyond the scope of what the gateway application should do.

i could probably add this feature in the main.py of my addon

FYI, I don't think your add-on would get approval to be added to the add-on list if it did this.

On reflection I think the current implementation is the right solution, so I'm going to close this issue.

— Reply to this email directly, view it on GitHub https://github.com/WebThingsIO/gateway/issues/2989#issuecomment-1294994983, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSPAAH6XYU4HNZLVNYOJJ3WFPHRDANCNFSM6AAAAAARLIAG4A . You are receiving this because you authored the thread.Message ID: @.***>

[benfrancis]

Your library suggests Python 3.10, but what is the minimum required version?

It looks like Buster currently ships 3.7.3 https://packages.debian.org/search?suite=oldstable&section=all&arch=any&searchon=names&keywords=python3

[arist0v]

3.10 minimum, and since lot of new en library would require higher python, going with 3.10 could open the door to more awesome addon

Le ven. 28 oct. 2022, 09 h 47, Ben Francis @.***> a écrit :

Your library suggests Python 3.10, but what is the minimum required version?

It looks like Buster currently ships 3.7.3 https://packages.debian.org/search?suite=oldstable&section=all&arch=any&searchon=names&keywords=python3

— Reply to this email directly, view it on GitHub https://github.com/WebThingsIO/gateway/issues/2989#issuecomment-1295022299, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSPAAHOXYSLYBFYL2VDR3TWFPKPPANCNFSM6AAAAAARLIAG4A . You are receiving this because you authored the thread.Message ID: @.***>

[benfrancis]

If I understand correctly in order to install Python 3.10 on Buster (Debian 10) you'd need to build it from source.

Even the current stable Bullseye (Debian 11) only comes with Python 3.9.2.

[arist0v]

that's insane that they didn't provide a more recent python version!!!

Le ven. 28 oct. 2022, à 10 h 05, Ben Francis @.***> a écrit :

If I understand correctly in order to install Python 3.10 on Buster (Debian 10) you'd need to build it from source.

Even the current stable Bullseye (Debian 11) only comes with Python 3.9.2.

— Reply to this email directly, view it on GitHub https://github.com/WebThingsIO/gateway/issues/2989#issuecomment-1295042333, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSPAAH4Y4GU6YEQY2DFEVTWFPMQXANCNFSM6AAAAAARLIAG4A . You are receiving this because you authored the thread.Message ID: @.***>

[benfrancis]

That's Debian for you :)

[arist0v]

I just think about that,

1 - you could package the gateway rpi image, and the docker image with the correct version of python(3.10) 2 - i neverd did rpm or deb packaging, but i'm pretty sure you could specify python 3.10 as dependency,

so you won'T have to handle python version in the gateway, the image and the docker will be provided with the correct version, and the deb. rpm will require the package manager/ the user to correct the missing depedency.

[benfrancis]

As far as I can tell Python 3.10 is not available in the Raspbian/Debian Buster repository upon which the WebThings Gateway raspberry Pi image is built.

Even if we built a future version of the image on Raspbian/Debian Bullseye (current stable) it could only use Python 3.9.2 and we have no way of automatically upgrading existing gateways to a new OS version (see #2801). If we build a future version on top of Bullseye, it's also likely the OTA updates of the gateway application would not work for existing gateways either.

For now add-ons must support Python 3.7.x if they want to run on WebThings Gateway 1.x, sorry about that.

[arist0v]

ok so i'll hope you could at least push the python version to 3.9