Latest lxml from streamlink 2.4.0 introduces build issues in Alpine (Docker)

[cgomesu]

Unable to install (or update to) streamlink 2.4.0 in Alpine (LinuxServer Docker image) due to a build issue with the latest version of lxml.

Log of the error:

[TVHlink] [info] Updating Streamlink...
Collecting streamlink
  Downloading streamlink-2.4.0-py3-none-any.whl (360 kB)
Requirement already satisfied, skipping upgrade: pycryptodome<4,>=3.4.3 in /usr/lib/python3.8/site-packages (from streamlink) (3.10.1)
Requirement already satisfied, skipping upgrade: iso-639 in /usr/lib/python3.8/site-packages (from streamlink) (0.4.5)
Requirement already satisfied, skipping upgrade: isodate in /usr/lib/python3.8/site-packages (from streamlink) (0.6.0)
Collecting lxml>=4.6.3
  Downloading lxml-4.6.3.tar.gz (3.2 MB)
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yrz_cxdn/lxml/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yrz_cxdn/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-2lo69xx4
         cwd: /tmp/pip-install-yrz_cxdn/lxml/
    Complete output (3 lines):
    Building lxml version 4.6.3.
    Building without Cython.
    Error: Please make sure the libxml2 and libxslt development packages are installed.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
[TVHlink] [error] Streamlink update failed!

[cgomesu]

Latest working version of Streamlink is 2.3.0 in Alpine v3.12 branch (armv7 and x86_64):

Package            Version
------------------ ----------
appdirs            1.4.4
CacheControl       0.12.6
certifi            2020.4.5.1
chardet            3.0.4
charset-normalizer 2.0.5
colorama           0.4.3
contextlib2        0.6.0
distlib            0.3.0
distro             1.5.0
html5lib           1.0.1
idna               2.9
iso-639            0.4.5
iso3166            2.0.1
isodate            0.6.0
lockfile           0.12.2
msgpack            1.0.0
ordered-set        4.0.1
packaging          20.4
pep517             0.8.2
pip                20.1.1
progress           1.5
pycryptodome       3.10.1
pyparsing          2.4.7
PySocks            1.7.1
pytoml             0.1.21
requests           2.26.0
retrying           1.3.3
setuptools         58.0.4
six                1.15.0
streamlink         2.3.0
toml               0.10.1
urllib3            1.25.9
webencodings       0.5.1
websocket-client   1.2.1

---

Streamlink 2.4.0 confirmed to install in Alpine edge branch (armv7 and x86_64):

Package            Version
------------------ ---------
appdirs            1.4.4
CacheControl       0.12.6
certifi            2020.12.5
chardet            4.0.0
charset-normalizer 2.0.5
colorama           0.4.4
contextlib2        21.6.0
distlib            0.3.2
distro             1.6.0
html5lib           1.1
idna               3.2
iso-639            0.4.5
iso3166            2.0.1
isodate            0.6.0
lockfile           0.12.2
lxml               4.6.3
msgpack            1.0.2
ordered-set        4.0.2
packaging          20.9
pep517             0.11.0
pip                20.3.4
progress           1.6
pycryptodome       3.10.1
pyparsing          2.4.7
PySocks            1.7.1
requests           2.26.0
retrying           1.3.3
setuptools         52.0.0
six                1.15.0
streamlink         2.4.0
toml               0.10.2
tomli              1.2.1
urllib3            1.26.6
webencodings       0.5.1
websocket-client   1.2.1

[cgomesu]

Issue is directly related to the addition of the lxml dependency mentioned in issue streamlink/streamlink#3944 and merged from the PR streamlink/streamlink#3952. Planning on taking a closer look at this over the weekend.

[cgomesu]

• created a discussion in the streamlink repo to document the issue: streamlink/streamlink#4036.

• building lxml 4.6.3 from source seems possible but it takes a long time (more than 30min) and requires more memory than a few (of my) hosts have available (more than 1GB)

• an alternative mentioned by @bastimeyer is to install lxml via the package manager apk instead of pip. the Docker image is on Alpine v3.12 and unfortunately, the latest version of the pkg py3-lxml in that branch is 4.5.1, which does not satisfy the streamlink 2.4.0 requirement that lxml>=4.6.3.

  • installing py3-lxml (4.6.3) from the edge branch adds the pkg lxml as a py39 package but v3.12 is on py38 instead. upgrading python adds another set of compatibility issues that need to be solved manually;
  • py3-lxml (4.6.3) from the latest-stable (v3.14) introduces the same issue as edge;
  • however, py3-lxml (4.6.3) from v3.13 installs as a py38 package. pip3 is able to recognize the package and its version satisfies the streamlink 2.4.0 requirement without the need to patch any source file.
    • streamlink successfully upgraded from 2.3.0 to 2.4.0 after applying this solution to one of my arm hosts.
    • will test the solution with another host architecture and if all looks good, will push a hotfix to https://github.com/cgomesu/tvhlink/blob/master/tools/docker/streamlink_for_tvh_container.sh later this week.