pip-generator fails for pyqt5

[ZanderBrown]

When running against the mu-editor package

Collecting pyqt5==5.10.1 (from mu-editor)
  Could not find a version that satisfies the requirement pyqt5==5.10.1 (from mu-editor) (from versions: )
No matching distribution found for pyqt5==5.10.1 (from mu-editor)

Or against pyqt5 directly

Collecting pyqt5
  Could not find a version that satisfies the requirement pyqt5 (from versions: )
No matching distribution found for pyqt5
Traceback (most recent call last):
  File "./flatpak-builder-tools/pip/flatpak-pip-generator", line 64, in <module>
    ] + opts.packages, check=True)
  File "/usr/lib64/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['pip3', 'download', '--dest', '/tmp/pip-generator-python3-pyqt5-kjvf0c4u', '--no-binary', ':all:', 'pyqt5']' returned non-zero exit status 1.

[TingPing]

Looks like pyqt5 simply only ships binary releases on pypi, this script explicitly avoids binary releases since well.. that isn't portable or useful for many flatpaks.

The good news is dozens of packages build pyqt from source that you can use as an example: https://github.com/search?q=org%3Aflathub+pyqt&type=Code

[ghisvail]

this script explicitly avoids binary releases since well.. that isn't portable or useful for many flatpaks.

Agreed, though I'd still want to get an initial generated-sources.json, including the problematic pyqt5 entry, to avoid writing them by hand. I can then remove the pyqt5 entry and add it to the manifest separately using one of your examples.

Right now, it's all or nothing sadly.

[ZanderBrown]

@ghisvail or just emit a warning and omit the entry? Largely anything but the current situation.

[ghisvail]

@ZanderBrown, you can't really omit the entry, since you still need potential leaf dependencies for these packages which don't provide an appropriate source release on PyPI.

I tried hacking a solution by fiddling with the arguments baked to the pip download subprocess, using the --only-binary option followed by a selected set of dependencies (pyqt5, sip...). The script still fails further down the line for the applications I have tried (mu-editor and spyder).

[ghisvail]

The ideal solution would be to have these projects adopt Pipfiles and commit a corresponding Pipfile.lock to the codebase. This way, the process of generating the corresponding generated-sources.json would be similar to what is done currently with Javascript.

[TingPing]

Pipfile support was on my todo list, it has nothing to do with this issue though. Using binaries at all is not a solution, period.

The best UX we can provide is manually generate a correct pyqt entry to build from source when it is detected.

[scx]

@ZanderBrown

If you would like to migrate to Qt 5.11, feel free to reuse my modules: https://github.com/scx/flathub/blob/6dda0c2b19205a0a2e1cc3bc777256c86a22fecf/python3-sip.yaml https://github.com/scx/flathub/blob/6dda0c2b19205a0a2e1cc3bc777256c86a22fecf/python3-qt5.yaml

[danyeaw]

I can reproduce this issue in the latest master version.

(.env) dan@dan-Librem-13-v2:~/Projects/flatpak-builder-tools/pip$ python flatpak-pip-generator pyqt5
Collecting pyqt5
  Downloading https://files.pythonhosted.org/packages/12/63/a7a32557fdce42b1ddf2353d9e2a576143376046ead0b1918fd4cfe01b90/PyQt5-5.13.1-5.13.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl (62.1MB)
     |████████████████████████████████| 62.2MB 188kB/s 
  Saved /tmp/pip-generator-python3-pyqt5-8lbye91x/PyQt5-5.13.1-5.13.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl
Collecting PyQt5_sip<13,>=4.19.19
  Downloading https://files.pythonhosted.org/packages/37/15/de717dffb8b7a9f6e976dcb3611b31f62213527f21734462fcdbd6744e43/PyQt5_sip-12.7.0-cp38-cp38-manylinux1_x86_64.whl (275kB)
     |████████████████████████████████| 276kB 922kB/s 
  Saved /tmp/pip-generator-python3-pyqt5-8lbye91x/PyQt5_sip-12.7.0-cp38-cp38-manylinux1_x86_64.whl
Successfully downloaded pyqt5 PyQt5-sip
ERROR: Could not find a version that satisfies the requirement PyQt5-5.13.1 (from versions: none)
ERROR: No matching distribution found for PyQt5-5.13.1
Failed to download pyqt5
Please fix the module manually in the generated file

[roipoussiere]

Maybe this is related, spyder analysis fails with:

[...]
Collecting pyxdg>=0.26
  Using cached pyxdg-0.26.tar.gz (57 kB)
  Saved /tmp/pip-generator-python3-spyder-h6jhiw_5/pyxdg-0.26.tar.gz
ERROR: Could not find a version that satisfies the requirement pyqt5<5.13 (from spyder) (from versions: 5.14.0, 5.14.1, 5.14.2)
ERROR: No matching distribution found for pyqt5<5.13 (from spyder)
Failed to download spyder
Please fix the module manually in the generated file

[hfiguiere]

For PyQt the recommended way is to use the baseapp. I don't thing it's solvable differently without a lot or effort.