search

SynoCommunity / spksrc

Cross compilation framework to create native packages for the Synology's NAS
https://synocommunity.com
Other
3.04k stars 1.23k forks source link

META: Python 3.10 / 3.11 package status #5043

Closed th0ma7 closed 1 month ago

th0ma7 commented 2 years ago

Python 3.10 package status

High-level python PR

Packages formally using python 3.x

PACKAGE Python 3.10 migration Published Python 3.11 migration Published 3.11 COMMENT
bazarr ✔️ ✔️ Follow-up update #5139 (@smaarn)
beets ✔️ ✔️ #4946
borgbackup ✔️ ✔️ #5048 (co-author @sunlei)
Fixed: https://github.com/borgbackup/borg/issues/6105
domoticz Version online uses python38
duplicity ✔️ ✔️ #5054
fishnet ❌️ ❌️ @mayweed Rewritten in Rust (removal #5216)
flexget ✔️ ✔️ #5176
homeassistant ✔️ ✔️ #5579, #5096, ARMv5 support pending: https://github.com/numpy/numpy/issues/20664
Looks like working based on testing https://github.com/SynoCommunity/spksrc/pull/5095/commits/fbd98191653d2138140c697454d41e82edbd64ab
Confirmed, fixed https://github.com/SynoCommunity/spksrc/pull/5155
mercurial ✔️ ✔️ #5045
octoprint ✔️ ✔️ #5217
plexpy-custom ✔️ Ready for migration but BUG with the package install, to remove?
rdiff-backup ✔️ ✔️ #5447
rutorrent ✔️ ✔️ Migration to DSM7 #5455, #5669, #5617
sabnzbd ✔️ ✔️ ✔️ ✔️ #5109 (@Safihre)
#5137
#5527
sickchill ✔️ ✔️ #5431 @miigotu
salt-master ✔️ ✔️ #5017 co-author: @arnebjarne
salt-minion ✔️ ✔️ #5017 co-author: @arnebjarne, #5145
tvheadend ✔️ ✔️
znc ✔️ ✔️ #4951, #5091

Packages formally using python 2.x (#4524)

PACKAGE python 3.10 migration DSM7 migration Python 3.10 Published Python 3.11 migration Published 3.11 COMMENT
deluge ✔️ ✔️ ✔️ #5728 #4153, #5202, #5398
ffsync ✔️ ✔️ #5942
haproxy
salt-master ✔️ ✔️ ✔️ #5017 co-author: @arnebjarne

Framework clean-up

apps dependent packages Removed COMMENT
python3 (cross, spk, native) none ✔️ #5316
python38 (cross, spk, native) none TODO to remove once all python3.x packages published
cross/asyncstdlib none ✔️ #5175
cross/borgbackup borgbackup ✔️ #5048
cross/cryptography duplicity n/a Mandatory to build abi3 wheels
cross/cryptography-legacy python2 TODO to remove once all python3.x packages published
cross/deluge deluge ✔️ #5398
cross/dtlssocket homeassistant N/A Unable to migrate to pip
cross/duplicity duplicity ✔️ #5054
cross/gevent ffsync
cross/greenlet ffsync, python38
cross/immutables python3, python38 TODO to remove once all python3.x packages published
cross/libtorrent deluge ✔️ #5398
cross/lxml python2, python3, python38
cross/m2crypto python2
cross/mercurial mercurial ✔️ #5045
cross/msgpack-python python2, python3, python38
cross/numpy homeassistant ✔️ #5096, ARMv5 support pending: https://github.com/numpy/numpy/issues/20664
Fixed https://github.com/SynoCommunity/spksrc/pull/5155
cross/octoprint octoprint ✔️ #5217
cross/pillow none ✔️ #5175
cross/pillow_py2 python2
cross/pocketsphinx none ✔️ #5175
cross/poetry none ✔️ #5508
cross/pyalsa python2
cross/pyaudio python2
cross/pycryptodome none ✔️ #5175
cross/pycryptodomex none ✔️ #5175
cross/pycurl python2, python3, python38
cross/PyNaCl none ✔️ #5175
cross/pyyaml python2, python3, python38
cross/pyzmq python2
cross/rencode deluge ✔️ #5398
cross/ruamel.yaml python3, python38 TODO to remove once all python3.x packages published
cross/sphinxbase cross/pocketsphinx ✔️ #5175
cross/stockfish-niklasf spk/fishnet ✔️ @mayweed Rewritten in Rust (removal #5216)
cross/ujson homeassistant ✔️ #5096, Solved: https://github.com/ultrajson/ultrajson/issues/496
cross/umemcache none ✔️ #5175
cross/uwsgi python2
cross/webrtcvad none ✔️ #5175
hgy59 commented 2 years ago

@th0ma7 octoprint cannot install the plugins due to permission restrictions for the pip module/install folder. any suggestions?

2022-01-13 21:37:20,165 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-13 21:37:20,169 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on ds-arm for SSDP
2022-01-13 21:37:21,012 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-13 21:37:22,577 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 0.41s
2022-01-13 21:37:23,919 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 0.31s
2022-01-13 21:37:23,938 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-13 21:37:23,954 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> no), --user flag needed -> no, virtual env -> yes
2022-01-13 21:37:23,955 - octoprint.util.pip - INFO - ==> pip ok -> NO!
2022-01-13 21:37:23,955 - octoprint.util.pip - ERROR - Cannot use this pip install, can't write to the install dir and also can't use --user for installing. Check your setup and the permissions on /volume1/@appstore/octoprint/env/lib/python3.10/site-packages.
2022-01-13 21:37:23,989 - octoprint.plugin - ERROR - Error while calling plugin softwareupdate
Traceback (most recent call last):
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugin/__init__.py", line 271, in call_plugin
    result = getattr(plugin, method)(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/__init__.py", line 1737, in wrapper
    return f(*args, **kwargs)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 224, in on_after_startup
    self._check_environment()
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/plugins/softwareupdate/__init__.py", line 362, in _check_environment
    and get_comparable_version(versions["pip"])
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 170, in get_comparable_version
    version_string = normalize_version(version_string)
  File "/volume1/@appstore/octoprint/share/OctoPrint/src/octoprint/util/version.py", line 211, in normalize_version
    if "-" in version:
TypeError: argument of type 'NoneType' is not iterable

Edit:

th0ma7 commented 2 years ago

That's odd, never faced that issue. And good thing it isn't yet released, leaving time to settle on this.

th0ma7 commented 2 years ago

Just a thought, I now wonder if the virtualenv didn't appeared to be installed by root user during the installation phase? Although I recall last time I tested homeassistant that many many other wheels where being installed after the fact with no problem. Perhaps something changed somehow...

I'll try to check this out over the week-end.

th0ma7 commented 2 years ago

octoprint cannot install the plugins due to permission restrictions for the pip module/install folder. any suggestions?

I wasn't able to reproduce... I was able to successfully install Bed-Visualizer from the Install new Plugins... option. Although I noticed that a particular version of setuptools was YANKED on pypi, perhaps your install ended-up using it?

Installing plugin "Bed Level Visualizer" from https://github.com/jneilliii/OctoPrint-BedLevelVisualizer/archive/master.zip...
/volume1/@appstore/octoprint/env/bin/python3 -m pip --disable-pip-version-check install file:///tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip --no-cache-dir
Processing /tmp/tmpgfzttzb2/OctoPrint-BedLevelVisualizer-master.zip
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: OctoPrint in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages/OctoPrint-1.7.2-py3.10.egg (from Bed-Visualizer==1.1.1) (1.7.2)
Requirement already satisfied: OctoPrint-FileCheck>=2021.2.23 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.2.23)
Requirement already satisfied: OctoPrint-FirmwareCheck>=2021.10.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.11)
Requirement already satisfied: OctoPrint-PiSupport>=2021.10.28 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.28)
Requirement already satisfied: markupsafe<2.0,>=1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.1)
Requirement already satisfied: markdown<3.2,>=3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.1.1)
Requirement already satisfied: regex!=2018.11.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2021.10.23)
Requirement already satisfied: wrapt<1.13,>=1.12.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.12.1)
Requirement already satisfied: flask<2,>=1.1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.4)
Requirement already satisfied: Jinja2<3,>=2.11.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.11.3)
Requirement already satisfied: Flask-Login<0.6,>=0.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.5.0)
Requirement already satisfied: Flask-Babel<2,>=1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: Flask-Assets<3,>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: werkzeug<2,>=1.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.1)
Requirement already satisfied: itsdangerous<2,>=1.1.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.0)
Requirement already satisfied: cachelib<0.2,>=0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.1)
Requirement already satisfied: PyYAML<6,>=5.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.4.1)
Requirement already satisfied: pyserial<4,>=3.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (3.5)
Requirement already satisfied: netaddr<0.9,>=0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.8.0)
Requirement already satisfied: watchdog==0.10.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.10.4)
Requirement already satisfied: sarge==0.1.6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.1.6)
Requirement already satisfied: netifaces<1,>=0.11 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.11.0)
Requirement already satisfied: pylru<2,>=1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.2.0)
Requirement already satisfied: pkginfo<2,>=1.7.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.7.1)
Requirement already satisfied: requests<3,>=2.26.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.26.0)
Requirement already satisfied: semantic_version<3,>=2.8.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.8.5)
Requirement already satisfied: psutil<6,>=5.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.8.0)
Requirement already satisfied: Click<8,>=7.1.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (7.1.2)
Requirement already satisfied: future<1,>=0.18.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.18.2)
Requirement already satisfied: websocket-client<1,>=0.59 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.59.0)
Requirement already satisfied: emoji<2,>=1.4.2 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.6.1)
Requirement already satisfied: sentry-sdk<2,>=1.3.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4.3)
Requirement already satisfied: filetype<2,>=1.0.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.0.8)
Requirement already satisfied: zipstream-new<1.2,>=1.1.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.1.8)
Requirement already satisfied: blinker<2,>=1.4 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.4)
Requirement already satisfied: feedparser<7,>=6.0.8 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.0.8)
Requirement already satisfied: tornado<7,>=6 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (6.1)
Requirement already satisfied: zeroconf<0.34,>=0.33 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (0.33.4)
Requirement already satisfied: immutabledict<3,>=2.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.2.1)
Requirement already satisfied: pathvalidate<3,>=2.4.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (2.5.0)
Requirement already satisfied: colorlog<6,>=5.0.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (5.0.1)
Requirement already satisfied: unidecode in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from OctoPrint->Bed-Visualizer==1.1.1) (1.3.2)
Requirement already satisfied: pathtools>=0.1.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from watchdog==0.10.4->OctoPrint->Bed-Visualizer==1.1.1) (0.1.2)
Requirement already satisfied: sgmllib3k in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from feedparser<7,>=6.0.8->OctoPrint->Bed-Visualizer==1.1.1) (1.0.0)
Requirement already satisfied: webassets>=2.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Assets<3,>=2.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0)
Requirement already satisfied: Babel>=2.3 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2.9.1)
Requirement already satisfied: pytz in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from Flask-Babel<2,>=1.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.3)
Requirement already satisfied: setuptools>=36 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from markdown<3.2,>=3.1->OctoPrint->Bed-Visualizer==1.1.1) (60.5.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2.0.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from requests<3,>=2.26.0->OctoPrint->Bed-Visualizer==1.1.1) (2021.10.8)
Requirement already satisfied: six in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from websocket-client<1,>=0.59->OctoPrint->Bed-Visualizer==1.1.1) (1.16.0)
Requirement already satisfied: ifaddr>=0.1.7 in /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (from zeroconf<0.34,>=0.33->OctoPrint->Bed-Visualizer==1.1.1) (0.1.7)
Building wheels for collected packages: Bed-Visualizer
Building wheel for Bed-Visualizer (setup.py): started
Building wheel for Bed-Visualizer (setup.py): finished with status 'done'
Created wheel for Bed-Visualizer: filename=Bed_Visualizer-1.1.1-py3-none-any.whl size=4182731 sha256=78edd206f54028eeb9218469443099aa8caaf5a231226448d2cf28619aeb095f
Stored in directory: /tmp/pip-ephem-wheel-cache-bul75zdr/wheels/b5/d6/23/c582f689507a9b7cf7420184f0fe2c5b595600fc122bac8917
Successfully built Bed-Visualizer
Installing collected packages: Bed-Visualizer
Successfully installed Bed-Visualizer-1.1.1
Done!

Would you mind uninstalling it / reinstalling it and see if the issue is still there? On my end I'll give it another shot using a fully clean tree on master to check if I can reproduce.

arch: kvmx64-7.0

EDIT: I'll also give it a shot on another arch such as armv7 / armv5 that I have available at home

EDIT2: Found the same log file as you referred, and all is good there as well (re-built from maser). Also still able to install an additional plugin:

2022-01-17 21:27:32,944 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:27:34,086 - octoprint.util.pip - INFO - Version of pip is 21.3.1
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:27:34,088 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:27:34,143 - octoprint.plugin.core - INFO - Initialized 17 plugin implementation(s)
2022-01-17 21:27:34,232 - octoprint.plugin.core - INFO - 17 plugin(s) registered with the system:

...
2022-01-17 21:31:35,501 - octoprint.plugins.pluginmanager - INFO - The plugin was installed successfully: Bed Visualizer, version 1.1.1
2022-01-17 21:31:35,893 - octoprint.plugins.tracking - INFO - Sent tracking event install_plugin, payload: {'plugin': 'bedlevelvisualizer', 'plugin_version': '1.1.1'}

EDIT 3: same on armv7, no issues:

2022-01-17 21:55:10,110 - octoprint.server.util.watchdog - INFO - ... initial scan done.
2022-01-17 21:55:11,037 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._http._tcp.local.' for _http._tcp.local.
2022-01-17 21:55:12,074 - octoprint.plugins.discovery - INFO - Registered 'OctoPrint instance on DS115j-armv7._octoprint._tcp.local.' for _octoprint._tcp.local.
2022-01-17 21:55:12,086 - octoprint.plugins.discovery - INFO - Registered OctoPrint instance on DS115j-armv7 for SSDP
2022-01-17 21:55:12,106 - octoprint.server - INFO - Listening on http://0.0.0.0:8088 and http://[::]:8088
2022-01-17 21:55:13,905 - octoprint.plugins.pluginmanager - INFO - Loaded plugin repository data from https://plugins.octoprint.org/plugins.json
2022-01-17 21:55:22,503 - octoprint.plugins.announcements - INFO - Loaded channel _important from https://octoprint.org/feeds/important.xml in 1.5s
2022-01-17 21:55:29,709 - octoprint.plugins.announcements - INFO - Loaded channel _releases from https://octoprint.org/feeds/releases.xml in 2.4s
2022-01-17 21:55:33,037 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:33,082 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:33,083 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:33,179 - octoprint.plugins.softwareupdate - INFO - Minimum free storage across all update relevant locations is 120.1MB. That is considered insufficient for updating.
2022-01-17 21:55:33,181 - octoprint.plugins.softwareupdate - INFO - Fetching check overlays from https://plugins.octoprint.org/update_check_overlay.json
2022-01-17 21:55:36,966 - octoprint.plugins.announcements - INFO - Loaded channel _blog from https://octoprint.org/feeds/octoblog.xml in 2.0s
2022-01-17 21:55:42,437 - octoprint.plugins.announcements - INFO - Loaded channel _plugins from https://plugins.octoprint.org/feed.xml in 1.4s
2022-01-17 21:55:47,198 - octoprint.plugins.announcements - INFO - Loaded channel _octopi from https://octoprint.org/feeds/octopi.xml in 1.2s
2022-01-17 21:55:48,419 - octoprint.plugins.pluginmanager - INFO - Loaded plugin notices data from https://plugins.octoprint.org/notices.json
2022-01-17 21:55:51,217 - octoprint.util.pip - INFO - Using "/volume1/@appstore/octoprint/env/bin/python3 -m pip" as command to invoke pip
2022-01-17 21:55:51,225 - octoprint.util.pip - INFO - pip installs to /volume1/@appstore/octoprint/env/lib/python3.10/site-packages (writable -> yes), --user flag needed -> no, virtual env -> yes
2022-01-17 21:55:51,226 - octoprint.util.pip - INFO - ==> pip ok -> yes
2022-01-17 21:55:51,354 - octoprint.plugins.softwareupdate - INFO - Saved version cache to disk
th0ma7 commented 2 years ago

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

miigotu commented 2 years ago

@miigotu the sickchill package is in theory ready for publishing. Was there any update planned that could benefit from being part of the python310 update?

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

Sickchill should work fine from 3.6.2 to python 3.11.x, and possibly newer as long as all of the dependencies that require c-extenstion/rust-extensions are built.

hgy59 commented 2 years ago

@hgy59 on a similar note, I haven't published homeassistant just yet. Are you planning for an update anything soon?

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel. (I am currently on vacation and have no access to my spk dev env).

th0ma7 commented 2 years ago

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

miigotu commented 2 years ago

Thnx for your feedbacks to both of you. So it looks like rust is mandatory for any next steps...

Yes, rust is going to become the defacto replacement for all c extensions over the next few years, and cryptography is a requirement of any application that makes web requests unless a pure python replacement comes along some day (crosses fingers). Cryptography could be done entirely in python with a small performance hit.

th0ma7 commented 2 years ago

rust is on my todo list but considering I've been sick over the last moth this will have to wait a little longer until I get better (or someone else tackle this over). In the meantime, we can stick with cryptography version 3.3.2 ...

th0ma7 commented 2 years ago

@mayweed is there any interest in upgrading fishnet? Current version uses an older python3 and is 1.18.1 while new version is at 2.5.1. EDIT: Note that I can help in getting there but I have no clue how to use this software.

mayweed commented 2 years ago

No, fishnet was rewritten in Rust actually. So it's a whole New package now...

Safihre commented 2 years ago

@mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

miigotu commented 2 years ago

No, fishnet was rewritten in Rust actually. So it's a whole New package now... @mayweed So for now we can just abandon it until someone updates it? I don't think we do rust yet..

I wonder if utilizing docker for pure rust builds is a good option for synocommunity?

FROM rust:1.59.0 AS builder
WORKDIR /fishnet
RUN mkdir /source
RUN git clone git@bitbucket.org:Pumalo/docker-conf.git /fichnet && (git submodule update --init --recursive || true) && cargo build --release -vv

FROM scratch as export-stage
RUN mkdir -p /output/$TARGETPLATFORM
COPY --from=builder /fishnet/fishnet /output/$TARGETPLATFORM/
name: Build all necessary architectures for fishnet

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1
    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1
    - name: Cache Docker layers
      uses: actions/cache@v2
      with:
        path: /tmp/.buildx-cache
        key: ${{ runner.os }}-buildx-${{ github.sha }}
        restore-keys: |
          ${{ runner.os }}-buildx-
    - name: Build docker image
      uses: docker/build-push-action@v2
      with:
        context: .
        platforms: linux/amd64,linux/arm64, <add all arch here>
        push: false
        output: type=local,dest=output_dir
        tags: fishnet-build
        cache-from: type=local,src=/tmp/.buildx-cache
        cache-to: type=local,dest=/tmp/.buildx-cache-new

You'd have all of the binaries for each platform in output_dir/$TARGETPLATFORM/fishnet, and then you add that to your spk however you want. I'm not sure how difficult it would be to offload some of the work to github actions in this way but it might be beneficial.

Maybe a third party repository to automatically build these binaries and add them as release artifacts, thatr synocommunnity can wget in the build process.

miigotu commented 2 years ago

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

hgy59 commented 2 years ago

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

miigotu commented 2 years ago

Working on a POC just to see if I can build rust binaries that way: https://github.com/miigotu/fishnet-binaries

We already have rust support in spksrc. The following (cli) tools are built with rust:

  • cross/bat
  • cross/dua
  • cross/dutree
  • cross/exa
  • cross/ripgrep

I know that, but I still want to see if I can do this thing that way lol

hgy59 commented 2 years ago

As I mentioned earlier, any homeassistant update needs cryptography ... that needs setuptools_rust to build the wheel.

@th0ma7 any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

th0ma7 commented 2 years ago

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

Safihre commented 2 years ago

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

hgy59 commented 2 years ago

Updated that python3 package was removed. Should we also remove/disable it on https://synocommunity.com/ ?

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

th0ma7 commented 2 years ago

There are still packages that depend on python3 (i.e. 3.7) in the repo, that are not updated to Python 3.10/3.8 yet

  • plexpy-custom
  • rdiff-backup
  • rutorrent (?)

@hgy59 good catch, while they where all migrated to python310 but not published yet. And as for python38, only one left is domoticz.

hogi1200 commented 2 years ago

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update 😄).

Nope, not yet. Nothing started neither.

@th0ma7 may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions? seems like everybody here thinks you are the right guy for this task. 🙏 😉

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel. but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

th0ma7 commented 2 years ago

Sadly my cycles are really limited at the moment and will remain for a few weeks. Realistically this might slip until end of summer early fall unless i stumble on an easy solution before then.

hogi1200 commented 2 years ago

@th0ma7 thanks for the quick response. Your contributions are still highly appreciated 👍

hgy59 commented 2 years ago

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

th0ma7 commented 2 years ago

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

Nope, and sadly no clue what taht is (on vacation with limited internet)

miigotu commented 2 years ago

@th0ma7 did you ever consider gateone package? (https://github.com/SynoCommunity/spksrc/issues/4524#issuecomment-1213032217)

Nope, and sadly no clue what taht is (on vacation with limited internet)

"Gate One is an HTML5-powered terminal emulator and SSH client"

https://github.com/liftoff/GateOne

th0ma7 commented 2 years ago

@cytec I tested the plexpy-custom package which makes use github.com/Tautulli/Tautulli.git Is there an interest at keeping this package alive and maintained? Thnx in advance.

th0ma7 commented 2 years ago

any progress on building cryptography (cross) wheel with rust? (homeassistant package needs an update smile).

Nope, not yet. Nothing started neither.

@th0ma7 may i ask if you already have a plan, when you will add rust support to enable cross compilation of newer cryptography versions? seems like everybody here thinks you are the right guy for this task. pray wink

my target is to be able to build a wheel of cryptography 36.0.2, so that I can upgrade my homeassistant core installation using that wheel. but i am still a noob to spksrc cross-compilation toolchain, so i kind of depend on a working framework.

@hogi1200 rust wheels is now available on master, enjoy :)

Casuallynoted commented 2 years ago

Any information on Mylar? Currently gives Python errors, I saw this Issue linked to an issue about Mylar but don't see it in the table in the original post.

hgy59 commented 2 years ago

@Casuallynoted you find the request for Mylar (3) in #4232

th0ma7 commented 1 year ago

@SynoCommunity/developers is there still interest in domoticz? It's the only py38 package remaining.

miigotu commented 1 year ago

@SynoCommunity/developers is there still interest in domoticz? It's the only py38 package remaining.

I had thought this was a Russian software, but I just learned it is from India. Honestly, there are much more advanced and widely used smart home operating systems and standalone softwares such as home assistant and zigbee2mqtt.

th0ma7 commented 1 year ago

@SynoCommunity/developers I've noticed that there is only domoticz that isn't yet migrated over to a newer version of python leaving python38 still in the code. Anyone interested in updating it or should it simply get removed entirely?

EDIT: Re-reding my previous post, if no one is against it I'll remove domoticz along with python38 entirely.

Safihre commented 1 year ago

You have my vote. Nobody bothered to fix it in years..

hgy59 commented 1 year ago

There is still a PR for domoticz (#4730) Since domoticz package does not depend on Python 3.8 package, we can remove Python 3.8 packages from the repo (after double checking that none of the published packages depend on python 3.8) But we should keep python38 in native and cross folders (until #4730 is updated to at least python 3.10)

th0ma7 commented 1 year ago

There is still a PR for domoticz (#4730)

That PR is 2 years old... you really intent to pursue that? Anyway, I'll revert my commit from #5724 for now.

EDIT: @hgy59 I ended-up updating python38 up to latest version until we get rid of it entirely. Let me know if I can help at completing domoticz migration towards 3.10-3.11 asap to clean-up older python2 and python38 along with the many cross/*

th0ma7 commented 1 month ago

Closing as since all packages got migrated to 3.11.