search

flatpak / flatpak-builder-tools

Various helper tools for flatpak-builder
198 stars 108 forks source link

flatpak-pip-generator not working for matplotlib #47

Open leinardi opened 5 years ago

leinardi commented 5 years ago

If I generate the json module for matplotlib using flatpak-pip-generator and than I try to use it it fails with a numpy error:

Running: pip3 install --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} matplotlib
Looking in links: file:///run/build/python3-matplotlib
Collecting matplotlib
Complete output from command python setup.py egg_info:
Download error on https://pypi.python.org/simple/numpy/: [Errno -3] Temporary failure in name resolution -- Some packages may not be found!
Couldn't find index page for 'numpy' (maybe misspelled?)
Download error on https://pypi.python.org/simple/: [Errno -3] Temporary failure in name resolution -- Some packages may not be found!
No local packages or working download links found for numpy>=1.10.0

If you manually add the command to install numpy if fails with python-dateutil not finding setuptools.

I managed to workaround this issue with this module configuration (not sure if the sed command is necessary):

{
  "name": "python-setuptools_scm",
  "buildsystem": "simple",
  "build-commands": [
    "python3 setup.py install --prefix=/app --root=/"
  ],
  "sources": [
    {
      "type": "archive",
      "url": "https://files.pythonhosted.org/packages/09/b4/d148a70543b42ff3d81d57381f33104f32b91f970ad7873f463e75bf7453/setuptools_scm-3.1.0.tar.gz",
      "sha256": "1191f2a136b5e86f7ca8ab00a97ef7aef997131f1f6d4971be69a1ef387d8b40"
    },
    {
      "type": "shell",
      "commands": [
        "sed -i 's/if not platform_/if not dirs/' setup.py"
      ]
    }
  ]
}
{
  "name": "python3-python-dateutil",
  "buildsystem": "simple",
  "build-commands": [
    "python3 setup.py install --prefix=/app --root=/"
  ],
  "sources": [
    {
      "type": "archive",
      "url": "https://files.pythonhosted.org/packages/0e/01/68747933e8d12263d41ce08119620d9a7e5eb72c876a3442257f74490da0/python-dateutil-2.7.5.tar.gz",
      "sha256": "88f9287c0174266bb0d8cedd395cfba9c58e87e5ad86b2ce58859bc11be3cf02"
    },
    {
      "type": "shell",
      "commands": [
        "sed -i 's/if not platform_/if not dirs/' setup.py"
      ]
    }
  ]
}
{
    "name": "python3-matplotlib",
    "buildsystem": "simple",
    "build-commands": [
        "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} numpy",
        "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} matplotlib"
    ],
    "sources": [
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/c2/4b/137dea450d6e1e3d474e1d873cd1d4f7d3beed7e0dc973b06e8e10d32488/cycler-0.10.0.tar.gz",
            "sha256": "cd7b2d1018258d7247a71425e9f26463dfb444d411c39569972f4ce586b0c9d8"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/31/60/494fcce70d60a598c32ee00e71542e52e27c978e5f8219fae0d4ac6e2864/kiwisolver-1.0.1.tar.gz",
            "sha256": "ce3be5d520b4d2c3e5eeb4cd2ef62b9b9ab8ac6b6fedbaa0e39cdb6f50644278"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/dd/bf/4138e7bfb757de47d1f4b6994648ec67a51efe58fa907c1e11e350cddfca/six-1.12.0.tar.gz",
            "sha256": "d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/0e/01/68747933e8d12263d41ce08119620d9a7e5eb72c876a3442257f74490da0/python-dateutil-2.7.5.tar.gz",
            "sha256": "88f9287c0174266bb0d8cedd395cfba9c58e87e5ad86b2ce58859bc11be3cf02"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/89/0c/653aec68e9cfb775c4fbae8f71011206e5e7fe4d60fcf01ea1a9d3bc957f/matplotlib-3.0.2.tar.gz",
            "sha256": "c94b792af431f6adb6859eb218137acd9a35f4f7442cea57e4a59c54751c36af"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/2d/80/1809de155bad674b494248bcfca0e49eb4c5d8bee58f26fe7a0dd45029e2/numpy-1.15.4.zip",
            "sha256": "3d734559db35aa3697dadcea492a423118c5c55d176da2f3be9c98d4803fc2a7"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/d0/09/3e6a5eeb6e04467b737d55f8bba15247ac0876f98fae659e58cd744430c6/pyparsing-2.3.0.tar.gz",
            "sha256": "f353aab21fd474459d97b709e527b5571314ee5f067441dc9f88e33eecd96592"
        }
    ]
}
danyeaw commented 5 years ago

I can't reproduce this issue with the latest master version. I think this is fixed by #63, since python_dateutil is now installed using the wheel.

leinardi commented 5 years ago

Sorry, I was planning to do some test during the weekend. I just tried and for me is still failing:

========================================================================
Building module python3-matplotlib in /home/leinardi/Workspace/gitlab/gwe/.flatpak-builder/build/python3-matplotlib-3
========================================================================
Running: pip3 install --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} matplotlib==3.1.1
Looking in links: file:///run/build/python3-matplotlib
Collecting matplotlib==3.1.1
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-llpb1jr6/matplotlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-llpb1jr6/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-llpb1jr6/matplotlib/
    Complete output (45 lines):
    Download error on https://pypi.org/simple/numpy/: [Errno -3] Temporary failure in name resolution -- Some packages may not be found!
    Couldn't find index page for 'numpy' (maybe misspelled?)
    Download error on https://pypi.org/simple/: [Errno -3] Temporary failure in name resolution -- Some packages may not be found!
    No local packages or working download links found for numpy>=1.11
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-llpb1jr6/matplotlib/setup.py", line 300, in <module>
        cmdclass=cmdclass,
      File "/usr/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/usr/lib/python3.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/lib/python3.7/site-packages/setuptools/dist.py", line 719, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/usr/lib/python3.7/site-packages/setuptools/dist.py", line 786, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
        raise DistutilsError(msg)
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('numpy>=1.11')
    ================================================================================
    Edit setup.cfg to change the build options

    BUILDING MATPLOTLIB
      matplotlib: yes [3.1.1]
          python: yes [3.7.4 (default, Nov 10 2011, 15:00:00)  [GCC 9.2.0]]
        platform: yes [linux]

    OPTIONAL SUBPACKAGES
     sample_data: yes [installing]
           tests: no  [skipping due to configuration]

    OPTIONAL BACKEND EXTENSIONS
             agg: yes [installing]
           tkagg: yes [installing; run-time loading from Python Tcl/Tk]
          macosx: no  [Mac OS-X only]

    OPTIONAL PACKAGE DATA
            dlls: no  [skipping due to configuration]

    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Error: module python3-matplotlib: Processo figlio uscito con codice 1

This is the pypi-dependencies.json:

{
    "name": "pypi-dependencies",
    "buildsystem": "simple",
    "build-commands": [],
    "modules": [
        {
            "name": "python3-injector",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} injector==0.17.0"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/0e/84/52df64867a7e9d3b690faeef505533333b3d2a67e3a9891465c40c2da0ba/injector-0.17.0-py2.py3-none-any.whl",
                    "sha256": "bb2179e218254ab51ffa5578fcb63cd93c6af250407ca51ec06060183208cede"
                }
            ]
        },
        {
            "name": "python3-matplotlib",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} matplotlib==3.1.1"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/7e/24/eaa8d7003aee23eda270099eeec754d7bf4399f75c6a011ef948304f66a2/pyparsing-2.4.2.tar.gz",
                    "sha256": "6f98a7b9397e206d78cc01df10131398f1c8b8510a2f4d97d9abd82e1aacdd80"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/c2/4b/137dea450d6e1e3d474e1d873cd1d4f7d3beed7e0dc973b06e8e10d32488/cycler-0.10.0.tar.gz",
                    "sha256": "cd7b2d1018258d7247a71425e9f26463dfb444d411c39569972f4ce586b0c9d8"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/12/d1/7b12cd79c791348cb0c78ce6e7d16bd72992f13c9f1e8e43d2725a6d8adf/matplotlib-3.1.1.tar.gz",
                    "sha256": "1febd22afe1489b13c6749ea059d392c03261b2950d1d45c17e3aed812080c93"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/16/e7/df58eb8868d183223692d2a62529a594f6414964a3ae93548467b146a24d/kiwisolver-1.1.0.tar.gz",
                    "sha256": "53eaed412477c836e1b9522c19858a8557d6e595077830146182225613b11a75"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/dd/bf/4138e7bfb757de47d1f4b6994648ec67a51efe58fa907c1e11e350cddfca/six-1.12.0.tar.gz",
                    "sha256": "d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/b6/d6/be8f975f5322336f62371c9abeb936d592c98c047ad63035f1b38ae08efe/numpy-1.17.3.zip",
                    "sha256": "a0678793096205a4d784bd99f32803ba8100f639cf3b932dc63b21621390ea7e"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl",
                    "sha256": "3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl",
                    "sha256": "1d8a5ae1ff6c5cf9b93e8811e581232ad8920aeec647c37316ceac982b08cb2d"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/ad/99/5b2e99737edeb28c71bcbec5b5dda19d0d9ef3ca3e92e3e925e7c0bb364c/python-dateutil-2.8.0.tar.gz",
                    "sha256": "c89805f6f4d64db21ed966fda138f8a5ed7a4fdbc1a8ee329ce1b74e3c74da9e"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl",
                    "sha256": "7e6584c74aeed623791615e26efd690f29817a27c73085b78e4bad02493df2fb"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl",
                    "sha256": "d9338df12903bbf5d65a0e4e87c2161968b10d2e489652bb47001d82a9b028b4"
                }
            ]
        },
        {
            "name": "python3-peewee",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} peewee==3.9.6"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/5e/66/5902f91abec78be8ef41eff4e547fe595c9e7b6c20f4be3d8cfb8e079061/peewee-3.9.6.tar.gz",
                    "sha256": "bc879e1ffb7f7684f90d9dd236c77b5042118d079f94764f35e3806948829ac8"
                }
            ]
        },
        {
            "name": "python3-py3nvml",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} py3nvml==0.2.3"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/19/4c/55e27f30541f7b32791764d21811d40fc28145cacf39cd9fd4c1b2452ecd/py3nvml-0.2.3-py3-none-any.whl",
                    "sha256": "ef106a134a1f8a2b172410741901c94c7e6c81a85066e24898e64e7ec7b354e2"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl",
                    "sha256": "8bbcb45cc982f48b2ca8fe7e7827c5d792f217ecf1792626f808bf41c3b86051"
                }
            ]
        },
        {
            "name": "python3-pygobject",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} pygobject"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/3c/1a/c0478ecab31baae50fda9956547788afbd0ca563adc52c9b03cab30f17eb/pycairo-1.18.2.tar.gz",
                    "sha256": "dcb853fd020729516e8828ad364084e752327d4cff8505d20b13504b32b16531"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/46/8a/b183f3edc812d4d28c8b671a922b5bc2863be5d38c56b3ad9155815e78dd/PyGObject-3.34.0.tar.gz",
                    "sha256": "2acb0daf2b3a23a90f52066cc23d1053339fee2f5f7f4275f8baa3704ae0c543"
                }
            ]
        },
        {
            "name": "python3-pyxdg",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} pyxdg==0.26"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/39/03/12eb9062f43adb94e30f366743cb5c83fd15fef026500cd4de42c7c12280/pyxdg-0.26-py2.py3-none-any.whl",
                    "sha256": "1948ff8e2db02156c0cccd2529b43c0cff56ebaa71f5f021bbd755bc1419190e"
                }
            ]
        },
        {
            "name": "python3-requests",
            "buildsystem": "simple",
            "build-commands": [
                "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} requests"
            ],
            "sources": [
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl",
                    "sha256": "3de946ffbed6e6746608990594d08faac602528ac7015ac28d33cee6a45b7398"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl",
                    "sha256": "ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl",
                    "sha256": "fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl",
                    "sha256": "9cf5292fcd0f598c671cfc1e0d7d1a7f13bb8085e9a590f48c010551dc6c4b31"
                },
                {
                    "type": "file",
                    "url": "https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl",
                    "sha256": "fd7c7c74727ddcf00e9acd26bba8da604ffec95bf1c2144e67aff7a8b50e6cef"
                }
            ]
        },
        {
            "name": "python3-Rx",
            "buildsystem": "simple",
            "build-commands": [
                "python3 setup.py install --prefix=/app --root=/"
            ],
            "sources": [
                {
                    "type": "archive",
                    "url": "https://files.pythonhosted.org/packages/9b/27/87ab7fc16a00141e0570ef8fc38c84f09e0840c9ce38ea7f1a79a9b32bcb/Rx-3.0.1.tar.gz",
                    "sha256": "c7d168618e3cec35fda9c9c8b5d7f966739df4b1a99b315eae1fc2d7d69e2512"
                },
                {
                    "type": "shell",
                    "commands": [
                    "sed -i \"/pytest\\-runner/d\" setup.py"
                    ]
                }
            ]
        }
    ]
}

I used the revision ed976db533e3552391a584396b9b321593fb1a39 of the flatpak-pip-generator.

@barthalion could you please re-open this issue?

barthalion commented 5 years ago

Sure thing.

LinkTed commented 4 years ago

Got the same error

LinkTed commented 4 years ago

It seems that the installation of numpy cause the problem. numpy tries to download additional files during the installation process. The following build options have to be added to have internet access during the installation process.

    "build-options": {
        "build-args": [ "--share=network" ]
    },

But if these lines are added to the json file, a new error appears:

ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-i6tnyi1h/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links file:///run/build/python3-numpy -- setuptools wheel 'Cython>=0.29.14'
       cwd: None
  Complete output (4 lines):
  Looking in links: file:///run/build/python3-numpy
  Collecting setuptools
    ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-i6tnyi1h/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links file:///run/build/python3-numpy -- setuptools wheel 'Cython>=0.29.14' Check the logs for full command output.
willprice commented 4 years ago

I can confirm the issue still happens with the latest version of flatpak-pip-generator from master with flatpak-builder 1.0.11 and matplotlib 3.3.2 (the latest), it tries to pull in numpy 1.19.2. Enabling --share=network gets passed the numpy requirements issue, but the build is still stuck at the cython/setuptools part:

flatpak-builder:2987796): Json-WARNING **: 17:20:19.102: Failed to deserialize "modules" property of type "gpointer" for an object of type "BuilderModule"
Emptying app dir 'build-dir'
Downloading sources
Starting build of uk.ac.bris.epic.narrator
Cache hit for python3-sounddevice, skipping build
Cache hit for python3-soundfile, skipping build
Cache hit for python3-python-vlc, skipping build
Cache miss, checking out last cache hit
========================================================================
Building module python3-matplotlib in /home/will/src/epic-kitchens-100-narrator/flatpak/.flatpak-builder/build/python3-matplotlib-5
========================================================================
Running: pip3 install --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "matplotlib==3.3.2"
Looking in links: file:///run/build/python3-matplotlib
Collecting matplotlib==3.3.2
Collecting certifi>=2020.06.20 (from matplotlib==3.3.2)
Collecting cycler>=0.10 (from matplotlib==3.3.2)
Collecting kiwisolver>=1.0.1 (from matplotlib==3.3.2)
Collecting numpy>=1.15 (from matplotlib==3.3.2)
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-poblqvi8/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links file:///run/build/python3-matplotlib -- 'setuptools<49.2.0' wheel 'Cython>=0.29.21'
cwd: None
Complete output (4 lines):
Looking in links: file:///run/build/python3-matplotlib
Collecting setuptools<49.2.0
ERROR: Could not find a version that satisfies the requirement setuptools<49.2.0 (from versions: none)
ERROR: No matching distribution found for setuptools<49.2.0
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-poblqvi8/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links file:///run/build/python3-matplotlib -- 'setuptools<49.2.0' wheel 'Cython>=0.29.21' Check the logs for full command output.
Error: module python3-matplotlib: Child process exited with code 1
willprice commented 4 years ago

Adding the additional dependencies setuptools, Cython and wheel enabled numpy to build. Here's what I ended up with for my matplotlib module:

{
    "name": "python3-matplotlib",
    "buildsystem": "simple",
    "build-commands": [
        "pip3 install --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} \"matplotlib==3.3.2\""
    ],
    "build-options": {
        "build-args": [ "--share=network" ]
    },
    "sources": [
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl",
            "sha256": "8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl",
            "sha256": "1d8a5ae1ff6c5cf9b93e8811e581232ad8920aeec647c37316ceac982b08cb2d"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/bf/e8/15aea783ea72e2d4e51e3ec365e8dc4a1a32c9e5eb3a6d695b0d58e67cdd/numpy-1.19.2.zip",
            "sha256": "0d310730e1e793527065ad7dde736197b705d0e4c9999775f212b03c44a8484c"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/62/b8/db619d97819afb52a3ff5ff6ad3f7de408cc83a8ec2dfb31a1731c0a97c2/kiwisolver-1.2.0.tar.gz",
            "sha256": "247800260cd38160c362d211dcaf4ed0f7816afb5efe56544748b21d6ad6d17f"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl",
            "sha256": "8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl",
            "sha256": "75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl",
            "sha256": "ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/3e/02/b09732ca4b14405ff159c470a612979acfc6e8645dc32f83ea0129709f7a/Pillow-7.2.0.tar.gz",
            "sha256": "97f9e7953a77d5a70f49b9a48da7776dc51e9b738151b22dacf101641594a626"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/2b/4c/fe4b36325795524f35d39edc390c89584e9a901df9e615df6f5effddaa0e/matplotlib-3.3.2.tar.gz",
            "sha256": "3d2edbf59367f03cd9daf42939ca06383a7d7803e3993eb5ff1bee8e8a3fbb6b"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/d0/4a/22ee76842d8ffc123d4fc48d24a623c1d206b99968fe3960039f1efc2cbc/setuptools-49.1.3.zip",
            "sha256": "ea484041dc6495c4ee74dd578470866e3043f6d1c998cfafbdb1f668f1768284"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/6c/9f/f501ba9d178aeb1f5bf7da1ad5619b207c90ac235d9859961c11829d0160/Cython-0.29.21.tar.gz",
            "sha256": "e57acb89bd55943c8d8bf813763d20b9099cc7165c0f16b707631a7654be9cad"
        },
        {
            "type": "file",
            "url": "https://files.pythonhosted.org/packages/83/72/611c121b6bd15479cb62f1a425b2e3372e121b324228df28e64cc28b01c2/wheel-0.35.1.tar.gz",
            "sha256": "99a22d87add3f634ff917310a3d87e499f19e663413a52eb9232c447aa646c9f"
        }
    ]
}
leinardi commented 4 years ago

@willprice this workaround seems to work locally but fails when you try to deploy it on Flathub:

]2;flatpak-builder: Building python3-matplotlib]2;flatpak-builder: Cleanup python3-matplotlibError: Can't specify build-args in sandboxed build
FB: unmounting rofiles-fuse /srv/buildbot/worker/build-x86_64-5/build/.flatpak-builder/rofiles/rofiles-49eGmy

https://flathub.org/builds/#/builders/35/builds/3610

aleb commented 3 years ago

Nowadays flatpak-pip-generator runs pip download to see what packages are needed, and then it downloads the source package for each of those. For example pip download downloads matplotlib-3.3.3-cp38-cp38-manylinux1_x86_64.whl and then flatpak-pip-generator goes out of its way to download and use matplotlib-3.3.3.tar.gz instead. The reason for downloading only source packages is security.

When using only source packages to build everything from source the build requirements are also needed. It seems pip is not aware about the build requirements though, so you need a few tries to know what build dependencies to pre-install and potentially the build dependencies of those etc.

Previously, an extra reason to download the sources was pip being run on the developer's system. Since flatpak-pip-generator now accepts the --runtime flag to run pip in a flatpak runtime, it's much easier to install the compiled packages:

$ flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 matplotlib

I hacked the tool to use exactly what pip download downloads initially (the compiled packages, when available): https://github.com/aleb/flatpak-builder-tools/commits/master

I generated flatpak manifest modules like this. 

python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 ipdb
python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 nose2
python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 pre-commit
python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 maplotlib
python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 nose setuptools_git setuptools_pep8 sphinx hotdoc
python3 flatpak-pip-generator --runtime org.gnome.Sdk/x86_64/3.38 pylint

Notice hotdoc which is available only as a source package required extra dependencies to be added manually when errors like these showed up: ERROR: No matching distribution found for sphinx.

danyeaw commented 3 years ago

For Gaphor we did something similar using a bash script:

https://www.github.com/flathub/org.gaphor.Gaphor/tree/master/depends.sh

aleb commented 3 years ago

A fresh attempt at building matplotlib from source on GNOME SDK 3.38: https://github.com/flathub/org.pitivi.Pitivi/pull/21#issuecomment-778584277

TL/DR: It's not possible because matplotlib 3.3.4 needs numpy 1.20.1, numpy needs an old setuptools, pip needs to uninstall the already-installed setuptools and fails: https://flathub.org/builds/#/builders/32/builds/39215

leinardi commented 3 years ago

Did anyone manage to build the latest matplotlib, v3.4.1?

gasinvein commented 3 years ago

@leinardi Does disabling build isolation and running the script inside flatpak sandbox help? Build isolation should be disabled by default since #169

leinardi commented 3 years ago

@gasinvein sorry, I'm not sure how to do that, if you can give me some guidance I'll be happy to test it.

So far I have generated the pypi-dependencies.json using the latest flatpak-pip-generator and you can see the result here: https://github.com/flathub/com.leinardi.gwe/blob/d2dd561c29e8c5a3e47ff7c68fed08cb8875b738/pypi-dependencies.json#L29

matplotlib has the --no-build-isolation flag, if this is what you wanted me to try. But I'm not sure what you mean by "running the script inside flatpak sandbox".

I get this error when i build the flatpak:

Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "matplotlib==3.4.1" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./matplotlib-3.4.1.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bocsk6t9/matplotlib_cc674f59331a46a8946e266c2bf0a919/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bocsk6t9/matplotlib_cc674f59331a46a8946e266c2bf0a919/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-e0wi8l40
         cwd: /tmp/pip-install-bocsk6t9/matplotlib_cc674f59331a46a8946e266c2bf0a919/
    Complete output (47 lines):
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe38a3f0790>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/numpy/
    WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe38a3f09a0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/numpy/
    WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe38a3f0b50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/numpy/
    WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe38a3f0d00>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/numpy/
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fe38a3f0eb0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/numpy/
    ERROR: Could not find a version that satisfies the requirement numpy>=1.16
    ERROR: No matching distribution found for numpy>=1.16
    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 119, in fetch_build_egg
        subprocess.check_call(cmd)
      File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpk9lo082k', '--quiet', 'numpy>=1.16']' returned non-zero exit status 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-bocsk6t9/matplotlib_cc674f59331a46a8946e266c2bf0a919/setup.py", line 258, in <module>
        setup(  # Finally, pass this all along to distutils to do the heavy lifting.
      File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 152, in setup
        _install_setup_requires(attrs)
      File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 686, in fetch_build_eggs
        resolved_dists = pkg_resources.working_set.resolve(
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 766, in resolve
        dist = best[req.key] = env.best_match(
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1051, in best_match
        return self.obtain(req, installer)
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1063, in obtain
        return installer(requirement)
      File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 745, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 121, in fetch_build_egg
        raise DistutilsError(str(e)) from e
    distutils.errors.DistutilsError: Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpk9lo082k', '--quiet', 'numpy>=1.16']' returned non-zero exit status 1.

    Edit setup.cfg to change the build options; suppress output with --quiet.

    BUILDING MATPLOTLIB
      matplotlib: yes [3.4.1]
          python: yes [3.8.8 (default, Nov 10 2011, 15:00:00)  [GCC 10.2.0]]
        platform: yes [linux]
           tests: no  [skipping due to configuration]
          macosx: no  [Mac OS-X only]

    ----------------------------------------
WARNING: Discarding file:///run/build/python3-matplotlib/matplotlib-3.4.1.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement matplotlib==3.4.1
ERROR: No matching distribution found for matplotlib==3.4.1
gasinvein commented 3 years ago

Hmm, looks like it wants numpy>=1.16, but for some reason doesn't like numpy-1.20.2 that was generated. Maybe it tries to install a version closer to 1.16? I'd try installing numpy first, then matplotlib. Try adding this command above the linked line:

"pip3 install --exists-action=i --no-index --find-links=\"file://${PWD}\" --prefix=${FLATPAK_DEST} \"numpy\" --no-build-isolation",

I'm not sure what you mean by "running the script inside flatpak sandbox"

Running flatpak-pip-generator script itself inside flatpak sandbox. I believe it has an option for this, --runtime or something.

leinardi commented 3 years ago

Unfortunately still fails:

Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "numpy" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./numpy-1.20.2.zip
    Preparing wheel metadata ... done
Building wheels for collected packages: numpy
  Building wheel for numpy (PEP 517) ... done
  Created wheel for numpy: filename=numpy-1.20.2-cp38-cp38-linux_x86_64.whl size=18390651 sha256=36521eaf93abfac18353847e871f9ca9133eb9a54dc3d9a64647bdf90433696a
  Stored in directory: /home/leinardi/.cache/pip/wheels/ea/bd/de/3bb9eb7e23bf035a2188fe279c169c449d5a2b4f96ff85ba63
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.20.2
Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "matplotlib==3.4.1" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./matplotlib-3.4.1.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-igbscpi9/matplotlib_fd12f64db4e94cbc97f8313c5d89816d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-igbscpi9/matplotlib_fd12f64db4e94cbc97f8313c5d89816d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-zg_v3ys9
         cwd: /tmp/pip-install-igbscpi9/matplotlib_fd12f64db4e94cbc97f8313c5d89816d/
    Complete output (47 lines):
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f114bf16820>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certifi/
    WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f114bf16a30>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certifi/
    WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f114bf16be0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certifi/
    WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f114bf16d90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certifi/
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f114bf16f40>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certifi/
    ERROR: Could not find a version that satisfies the requirement certifi>=2020.06.20
    ERROR: No matching distribution found for certifi>=2020.06.20
    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 119, in fetch_build_egg
        subprocess.check_call(cmd)
      File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp7oeul8rn', '--quiet', 'certifi>=2020.06.20']' returned non-zero exit status 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-igbscpi9/matplotlib_fd12f64db4e94cbc97f8313c5d89816d/setup.py", line 258, in <module>
        setup(  # Finally, pass this all along to distutils to do the heavy lifting.
      File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 152, in setup
        _install_setup_requires(attrs)
      File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 686, in fetch_build_eggs
        resolved_dists = pkg_resources.working_set.resolve(
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 766, in resolve
        dist = best[req.key] = env.best_match(
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1051, in best_match
        return self.obtain(req, installer)
      File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1063, in obtain
        return installer(requirement)
      File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 745, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 121, in fetch_build_egg
        raise DistutilsError(str(e)) from e
    distutils.errors.DistutilsError: Command '['/usr/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp7oeul8rn', '--quiet', 'certifi>=2020.06.20']' returned non-zero exit status 1.

    Edit setup.cfg to change the build options; suppress output with --quiet.

    BUILDING MATPLOTLIB
      matplotlib: yes [3.4.1]
          python: yes [3.8.8 (default, Nov 10 2011, 15:00:00)  [GCC 10.2.0]]
        platform: yes [linux]
           tests: no  [skipping due to configuration]
          macosx: no  [Mac OS-X only]

    ----------------------------------------
WARNING: Discarding file:///run/build/python3-matplotlib/matplotlib-3.4.1.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement matplotlib==3.4.1
ERROR: No matching distribution found for matplotlib==3.4.1
gasinvein commented 3 years ago

Well, now the error is different, now it wants certifi. I wonder why it wasn't generated. Maybe just install cerifi before matplotlib, then?

leinardi commented 3 years ago

I added it, now it looks for cppy>=1.1.0... I'll try to provide that as well...

leinardi commented 3 years ago

Now I'm stuck here:

Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "certifi" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./certifi-2020.12.5.tar.gz
Building wheels for collected packages: certifi
  Building wheel for certifi (setup.py) ... done
  Created wheel for certifi: filename=certifi-2020.12.5-py2.py3-none-any.whl size=147527 sha256=4693eed29ca2f17a34ea619b722e263452789806643e055cca137e8ae36421d0
  Stored in directory: /home/leinardi/.cache/pip/wheels/a6/e6/cb/1f615a05af0916f7a8149ef66f418e25104999cf30bc8710e9
Successfully built certifi
Installing collected packages: certifi
Successfully installed certifi-2020.12.5
Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "cppy" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./cppy-1.1.0.tar.gz
Building wheels for collected packages: cppy
  Building wheel for cppy (setup.py) ... done
  Created wheel for cppy: filename=cppy-1.1.0-py3-none-any.whl size=5748 sha256=0df3b12a28c932ba796a510c70129484318b628e59b5f3a16473f91cd36b781c
  Stored in directory: /home/leinardi/.cache/pip/wheels/93/e3/42/610c2b4617ba301113a36573c71b096a8d8ff2b83d280e2a27
Successfully built cppy
Installing collected packages: cppy
Successfully installed cppy-1.1.0
Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "numpy" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./numpy-1.20.2.zip
    Preparing wheel metadata ... done
Building wheels for collected packages: numpy
  Building wheel for numpy (PEP 517) ... done
  Created wheel for numpy: filename=numpy-1.20.2-cp38-cp38-linux_x86_64.whl size=18390701 sha256=861bf5d830c8f885fa6b39fb8a524e4a5fd9bd60f2cc41fc676ff6c1f9af4c53
  Stored in directory: /home/leinardi/.cache/pip/wheels/ea/bd/de/3bb9eb7e23bf035a2188fe279c169c449d5a2b4f96ff85ba63
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.20.2
Running: pip3 install --exists-action=i --no-index --find-links="file://${PWD}" --prefix=${FLATPAK_DEST} "matplotlib==3.4.1" --no-build-isolation
Looking in links: file:///run/build/python3-matplotlib
Processing ./matplotlib-3.4.1.tar.gz
Processing ./cycler-0.10.0.tar.gz
Processing ./kiwisolver-1.3.1.tar.gz
Requirement already satisfied: numpy>=1.16 in /app/lib/python3.8/site-packages (from matplotlib==3.4.1) (1.20.2)
Processing ./Pillow-8.2.0.tar.gz
Processing ./pyparsing-2.4.7.tar.gz
Requirement already satisfied: python-dateutil>=2.7 in /app/lib/python3.8/site-packages (from matplotlib==3.4.1) (2.8.1)
Requirement already satisfied: six in /usr/lib/python3.8/site-packages (from cycler>=0.10->matplotlib==3.4.1) (1.15.0)
Building wheels for collected packages: matplotlib, cycler, kiwisolver, pillow, pyparsing
  Building wheel for matplotlib (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ov12t7lw
       cwd: /tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/
  Complete output (552 lines):

  Edit setup.cfg to change the build options; suppress output with --quiet.

  BUILDING MATPLOTLIB
    matplotlib: yes [3.4.1]
        python: yes [3.8.8 (default, Nov 10 2011, 15:00:00)  [GCC 10.2.0]]
      platform: yes [linux]
         tests: no  [skipping due to configuration]
        macosx: no  [Mac OS-X only]

  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  copying lib/pylab.py -> build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/mpl_toolkits
  copying lib/mpl_toolkits/__init__.py -> build/lib.linux-x86_64-3.8/mpl_toolkits
  creating build/lib.linux-x86_64-3.8/matplotlib
  copying lib/matplotlib/widgets.py -> build/lib.linux-x86_64-3.8/matplotlib
[...]
  UPDATING build/lib.linux-x86_64-3.8/matplotlib/_version.py
  set build/lib.linux-x86_64-3.8/matplotlib/_version.py to '3.4.1'
  running build_ext
  creating tmp
  x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmppu3xaiiv.cpp -o tmp/tmppu3xaiiv.o -fvisibility=hidden
  x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmpaqpzqu2n.cpp -o tmp/tmpaqpzqu2n.o -fvisibility-inlines-hidden
  x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmp_axfgey8.cpp -o tmp/tmp_axfgey8.o -flto
  error: Failed to download any of the following: ['https://downloads.sourceforge.net/project/freetype/freetype2/2.6.1/freetype-2.6.1.tar.gz', 'https://download.savannah.gnu.org/releases/freetype/freetype-2.6.1.tar.gz'].  Please download one of these urls and extract it into 'build/' at the top-level of the source repository.
  ----------------------------------------
  ERROR: Failed building wheel for matplotlib
  Running setup.py clean for matplotlib
  Building wheel for cycler (setup.py) ... done
  Created wheel for cycler: filename=cycler-0.10.0-py3-none-any.whl size=6663 sha256=1f72bf13a47e24af8c326c03b888a51d589cfc722cb24a51a43d02519b46f4d2
  Stored in directory: /home/leinardi/.cache/pip/wheels/ac/0e/33/8da5ddb54cf52b7de0c8d8e3814c99ea7981416c21f922420b
  Building wheel for kiwisolver (setup.py) ... done
  Created wheel for kiwisolver: filename=kiwisolver-1.3.1-cp38-cp38-linux_x86_64.whl size=1177547 sha256=4f64b37f1efdc0bd641d9fc4687402340ac0b8d211899e73be5a27f8d3ffeb83
  Stored in directory: /home/leinardi/.cache/pip/wheels/94/1b/a4/fb31fac67a78aed610f010d8049868efe0d5939eecb9a2321a
  Building wheel for pillow (setup.py) ... done
  Created wheel for pillow: filename=Pillow-8.2.0-cp38-cp38-linux_x86_64.whl size=1211425 sha256=8468ed09eb22aa3cf02a8a547e1f4c31be3eaed7abb3bf90167aa613e1bdf366
  Stored in directory: /home/leinardi/.cache/pip/wheels/02/f9/e2/8d57e6646c6b3e4d28ce3ce5e15c34f036a7058e685e12d964
  Building wheel for pyparsing (setup.py) ... done
  Created wheel for pyparsing: filename=pyparsing-2.4.7-py2.py3-none-any.whl size=67843 sha256=3c1bbf4f23a17aa3e5a4f05ba9a6aff6727cf02527f70317feb0d21b264c534f
  Stored in directory: /home/leinardi/.cache/pip/wheels/68/fb/39/6ccd071b6ee9a6c8ea045c28d565991e58c2af01a5789d7330
Successfully built cycler kiwisolver pillow pyparsing
Failed to build matplotlib
Installing collected packages: pyparsing, pillow, kiwisolver, cycler, matplotlib
    Running setup.py install for matplotlib ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-63enffx6/install-record.txt --single-version-externally-managed --prefix /app --compile --install-headers /app/include/python3.8/matplotlib
         cwd: /tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/
    Complete output (551 lines):

    Edit setup.cfg to change the build options; suppress output with --quiet.

    BUILDING MATPLOTLIB
      matplotlib: yes [3.4.1]
          python: yes [3.8.8 (default, Nov 10 2011, 15:00:00)  [GCC 10.2.0]]
        platform: yes [linux]
           tests: no  [skipping due to configuration]
          macosx: no  [Mac OS-X only]

    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    copying lib/pylab.py -> build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/mpl_toolkits
    copying lib/mpl_toolkits/__init__.py -> build/lib.linux-x86_64-3.8/mpl_toolkits
    creating build/lib.linux-x86_64-3.8/matplotlib
    copying lib/matplotlib/widgets.py -> build/lib.linux-x86_64-3.8/matplotlib
    copying lib/matplotlib/units.py -> build/lib.linux-x86_64-3.8/matplotlib
    copying lib/matplotlib/type1font.py -> build/lib.linux-x86_64-3.8/matplotlib
    copying lib/matplotlib/ttconv.py -> build/lib.linux-x86_64-3.8/matplotlib
[...]
    UPDATING build/lib.linux-x86_64-3.8/matplotlib/_version.py
    set build/lib.linux-x86_64-3.8/matplotlib/_version.py to '3.4.1'
    running build_ext
    x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmpzjvfcaqg.cpp -o tmp/tmpzjvfcaqg.o -fvisibility=hidden
    x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmp56riqvla.cpp -o tmp/tmp56riqvla.o -fvisibility-inlines-hidden
    x86_64-unknown-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.8 -I/usr/include/x86_64-linux-gnu/python3.8 -c /tmp/tmp3px1su4o.cpp -o tmp/tmp3px1su4o.o -flto
    error: Failed to download any of the following: ['https://downloads.sourceforge.net/project/freetype/freetype2/2.6.1/freetype-2.6.1.tar.gz', 'https://download.savannah.gnu.org/releases/freetype/freetype-2.6.1.tar.gz'].  Please download one of these urls and extract it into 'build/' at the top-level of the source repository.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e0ykknwi/matplotlib_330a2763216844a0ad24718a340f8b37/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-63enffx6/install-record.txt --single-version-externally-managed --prefix /app --compile --install-headers /app/include/python3.8/matplotlib Check the logs for full command output.
gasinvein commented 3 years ago

@leinardi It's trying to download freetype source tarball. You can feed matplotlib a setup.cfg forcing it to use system freetype. In the meantime, I've managed to build it. Should I submit a PR?

leinardi commented 3 years ago

Should I submit a PR?

That would be awesome! Yes please! :smile:

leinardi commented 3 years ago

Thanks a lot! It works!

@aleb this might interest you as well.

gasinvein commented 3 years ago

@leinardi NP. I guess now we should figure out why certifi and cppy didn't get into generated-sources automatically.

SteffenBrinckmann commented 2 years ago

Hey, I am kind of lost. Is there already semi-automatic solution for including matplotlib or a workaround?