search

johannesjh / req2flatpak

Convert Python package requirements to Flatpak build manifests
https://johannesjh.github.io/req2flatpak/
MIT License
17 stars 2 forks source link

TypeError: '<' not supported between instances of 'NoneType' and 'Download' #70

Closed EvilSupahFly closed 1 day ago

EvilSupahFly commented 5 days ago

After changing requirements.txt, I get this:

Traceback (most recent call last):
  File "/home/evilsupahfly/src/Flatpak Testing/req2flatpak.py", line 831, in <module>
    main()
  File "/home/evilsupahfly/src/Flatpak Testing/req2flatpak.py", line 826, in main
    FlatpakGenerator.build_module_as_str(requirements, downloads)
  File "/home/evilsupahfly/src/Flatpak Testing/req2flatpak.py", line 643, in build_module_as_str
    return json.dumps(cls.build_module(*args, **kwargs), indent=4)
  File "/home/evilsupahfly/src/Flatpak Testing/req2flatpak.py", line 632, in build_module
    "sources": sources(downloads),
  File "/home/evilsupahfly/src/Flatpak Testing/req2flatpak.py", line 624, in sources
    return [source(download) for download in sorted(downloads)]
TypeError: '<' not supported between instances of 'NoneType' and 'Download'

I don't know if it will help, but my requirements.txt is as follows:

amulet-core==1.9
amulet-map-editor==0.10
amulet-nbt==2.0
cython==3.0.0a9
minecraft-resource-pack==1.3
numpy==1.17
packaging==21.3
pillow==10.0.1
platformdirs==3.1
pre-commit==1.11.1
pymctranslate==1.2
pyopengl==3.0
setuptools==42
sphinx==1.7.4
sphinx-autodoc-typehints==1.3.0
sphinx-rtd-theme==0.3.1
versioneer==0.29
wheel==0.37.1
wxpython==4.2.1
cbm755 commented 3 days ago

I was able to reproduce this:

$ eq2flatpak --requirements-file requirements.txt  --target-platform 312-x86_64
Traceback (most recent call last):
  File "/home/cbm/.local/bin/req2flatpak", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 826, in main
    FlatpakGenerator.build_module_as_str(requirements, downloads)
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 643, in build_module_as_str
    return json.dumps(cls.build_module(*args, **kwargs), indent=4)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 632, in build_module
    "sources": sources(downloads),
               ^^^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 624, in sources
    return [source(download) for download in sorted(downloads)]
                                             ^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'NoneType' and 'Download'

I guess one of these lines isn't valid somehow, but surely we could have a better error message...

cbm755 commented 3 days ago

I started pruning down the requirements.txt to a minimal example:

pyopengl==3.0

which gives:

$ req2flatpak --requirements-file requirements.txt  --target-platform 310-x86_64
Traceback (most recent call last):
  File "/home/cbm/.local/bin/req2flatpak", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 826, in main
    FlatpakGenerator.build_module_as_str(requirements, downloads)
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 643, in build_module_as_str
    return json.dumps(cls.build_module(*args, **kwargs), indent=4)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 632, in build_module
    "sources": sources(downloads),
               ^^^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 624, in sources
    return [source(download) for download in sorted(downloads)]
            ^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 616, in source
    "url": download.url,
           ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'url'

There is no 3.0 only 3.0.0 and that's from 2009. Likely not available for Python 3.10.

cbm755 commented 3 days ago

I think req2flatpak can improve the error messaging here though. While shortening the provided example, I also saw a third different error:

  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 245, in __lt__
    return sort_keys(self) < sort_keys(other)
                             ^^^^^^^^^^^^^^^^
  File "/home/cbm/.local/lib/python3.12/site-packages/req2flatpak.py", line 243, in sort_keys
    return download.package, download.version, download.arch or ""
           ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'package'
cbm755 commented 3 days ago

Maybe this is a reasonable fix?

diff --git a/req2flatpak.py b/req2flatpak.py
index 5982c04..b079720 100755
--- a/req2flatpak.py
+++ b/req2flatpak.py
@@ -502,6 +502,8 @@ class PypiClient:
         url = f"https://pypi.org/pypi/{req.package}/{req.version}/json"
         json_string = cls._query(url)
         json_dict = json.loads(json_string)
+        if not json_dict["urls"]:
+            raise RuntimeError(f"{req} has no URLs: maybe old?")
         return Release(
             package=req.package,
             version=req.version,
EvilSupahFly commented 2 days ago

Ah, see, in the original project, the requirements weren't frozen for pyOpenGL, it was ">=" not "==" and because there are technically versions higher than 3.0 available, even if 3.0 itself doesn't exist, there was never a complaint from PIP before.

Double-checking pip index versions pyopengl gives me this list:

WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
pyopengl (3.1.7)
Available versions: 3.1.7, 3.1.6, 3.1.5, 3.1.4, 3.1.0, 3.0.2, 3.0.1
  INSTALLED: 3.1.6
  LATEST:    3.1.7

So I set the version to 3.0.1 for my requirements to work, and it looks like it was successful:

[~/src/Flatpak Testing/req2flatpak] python3 req2flatpak.py --requirements-file requirements.txt --target-platforms 310-x86_64
{
    "name": "python3-package-installation",
    "buildsystem": "simple",
    "build-commands": [
        "pip3 install --verbose --exists-action=i --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} --no-build-isolation amulet-map-editor amulet-core amulet-nbt cython minecraft-resource-pack numpy packaging pillow platformdirs pre-commit pymctranslate pyopengl setuptools sphinx sphinx-autodoc-typehints sphinx-rtd-theme versioneer wheel wxpython"
    ],
    "sources": [
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/ce/08/a1202c0ae30098e7d7efc5d6755cee4ed0be6db5432c9b1476a5d1e3b3aa/amulet_core-1.9.0-py3-none-any.whl",
            "sha256": "a424f09762d9e205bad1e36198b365fe89fec0b74db89cb44b141dff616edfeb"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/be/e5/462e9eef00ababf5ce28690441ed25a09affe6d632e017d94d9a8e2fa1ce/amulet-map-editor-0.10.0.tar.gz",
            "sha256": "d53d71b1d071a033987f574c47f4c1ebc802120d4e6d7cf4ba833478c1a25785"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/9f/df/9c9c2952550c190a791f1dbe98e78c6bc9695397b75ce79d760816b85a56/amulet-nbt-2.0.0.tar.gz",
            "sha256": "f38f6d68daefd35207bade2b1c6f603d676173578846aeaf1c12bd822305a08e"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/27/8a/bdfcf6ba4ba13c4343c2183b8d4fb25d59b55ec8bfa41ddb80452c28c380/Cython-3.0.0a9-py2.py3-none-any.whl",
            "sha256": "d5a17108c5d765bacb7a7c16d339172e38379023746bb9126b9912086e7487e4"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/b0/6a/7cd59bf49e90ae254beabe45c377f3fd5b3f11bd194370bc0e40cb0ff88d/minecraft_resource_pack-1.3.0-py3-none-any.whl",
            "sha256": "453bd3fc6e18daa9f719cbf1500369158deaccf5e1eaa5204d05abf8de29ed0d"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/da/32/1b8f2bb5fb50e4db68543eb85ce37b9fa6660cd05b58bddfafafa7ed62da/numpy-1.17.0.zip",
            "sha256": "951fefe2fb73f84c620bec4e001e80a80ddaa1b84dce244ded7f1e0cbe0ed34a"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl",
            "sha256": "ef103e05f519cdc783ae24ea4e2e0f508a9c99b2d4969652eed6a2e1ea5bd522"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/7a/07/e896b096a77375e78e02ce222ae4fd6014928cd76c691d312060a1645dfa/Pillow-10.0.1-cp310-cp310-manylinux_2_28_x86_64.whl",
            "sha256": "0462b1496505a3462d0f35dc1c4d7b54069747d65d00ef48e736acda2c8cbdff",
            "only-arches": [
                "x86_64"
            ]
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/ca/de/a33823fe54d52ea72fdae011115d737a2642d441c93b68ed17455a328e4c/platformdirs-3.1.0-py3-none-any.whl",
            "sha256": "13b08a53ed71021350c9e300d4ea8668438fb0046ab3937ac9a29913a1a1350a"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/29/2f/0b7a48ce1895ad926c6d69f2e6eab66992ab4d6fd8c7000f2fe34030a36f/pre_commit-1.11.1-py2.py3-none-any.whl",
            "sha256": "e7efe81063b8cb2a72190db06109191aac67971fc7a31d9e9485907f6be76954"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/27/47/24d468040d342df3f27060ffd5830a7702f8eb8ab9f4d9c98ee8803ce55d/PyMCTranslate-1.2.0-py3-none-any.whl",
            "sha256": "a0bc2510e5f8ef83fe4af8701314ae7b8d616cd47286fa2ee2ecf42bac40116e"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/9c/1d/4544708aaa89f26c97cc09450bb333a23724a320923e74d73e028b3560f9/PyOpenGL-3.1.0.tar.gz",
            "sha256": "9b47c5c3a094fa518ca88aeed35ae75834d53e4285512c61879f67a48c94ddaf"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/f9/73/a1e8254ee8604a7f8598f5b7c09d587b329ad2727f410a210f26c9595b7a/setuptools-60.0.0-py3-none-any.whl",
            "sha256": "75fd345a47ce3d79595b27bf57e6f49c2ca7904f3c7ce75f8a87012046c86b0b"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/89/44/73cd04b856fa35a69e9e2e790aa283e2eaefb684e116f1d46d01a5e7986f/Sphinx-1.7.4-py2.py3-none-any.whl",
            "sha256": "2e7ad92e96eff1b2006cf9f0cdb2743dacbae63755458594e9e8238b0c3dc60b"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/e1/c2/33f27e8c4dc7f84ec5873e39a0ded7e90b85b1d6dfbcce28b6b40863ddbd/sphinx_autodoc_typehints-1.3.0-py3-none-any.whl",
            "sha256": "46cc9e985ee6d8bbbd07fffd95b815c39a72df6afb600f59671f85f7340e7d0d"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/47/33/e3a1cc08acf0c50418c505d1b954adc950141ace691e3b4a22ba91950a32/sphinx_rtd_theme-0.3.1-py2.py3-none-any.whl",
            "sha256": "32424dac2779f0840b4788fbccb032ba2496c1ca47a439ad2510c8b1e55dfd33"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/b0/79/f0f1ca286b78f6f33c521a36b5cbd5bd697c0d66217d8856f443aeb9dd77/versioneer-0.29-py3-none-any.whl",
            "sha256": "0f1a137bb5d6811e96a79bb0486798aeae9b9c6efc24b389659cebb0ee396cb9"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl",
            "sha256": "4bdcd7d840138086126cd09254dc6195fb4fc6f01c050a1d7236f2630db1d22a"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/aa/64/d749e767a8ce7bdc3d533334e03bb1106fc4e4803d16f931fada9007ee13/wxPython-4.2.1.tar.gz",
            "sha256": "e48de211a6606bf072ec3fa778771d6b746c00b7f4b970eb58728ddf56d13d5c"
        }
    ]
}
cbm755 commented 2 days ago

see docs but I think its --out ...

johannesjh commented 1 day ago

we merged #71, so the special case that was encountered in this issue, namely that pypi may not return any URLs for a given requirements specification, is now handled with an improved error message. I think we can close this ticket.

EvilSupahFly commented 1 day ago

🙂👍