open-reaction-database / ord-schema

Schema for the Open Reaction Database
https://open-reaction-database.org
Apache License 2.0
92 stars 26 forks source link

Psycopg2 dependency fails to compile in CI/CD #650

Closed d-miketa closed 1 year ago

d-miketa commented 1 year ago

Hi,

I use a CI/CD system for a package that lists ord-schema as a dependency. One of its subdependencies, psycopg2, is available on PyPI as source code that must be compiled by the local machine. Some of the compilation dependencies are not covered by PyPI, so without intervention the CI/CD pipeline fails:

Collecting psycopg2>=2.8.5
  Downloading psycopg2-2.9.4.tar.gz (384 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 384.0/384.0 kB 83.1 MB/s eta 0:00:00
  Preparing metadata ([setup.py](http://setup.py/)): started
  Preparing metadata ([setup.py](http://setup.py/)): finished with status 'error'
  error: subprocess-exited-with-error

  × python [setup.py]
 egg_info did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      /opt/pipelines/agent/build/.cache/nox/typing_check/lib/python3.9/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running egg_info
      creating /tmp/pip-pip-egg-info-vyczfsiy/psycopg2.egg-info
      writing /tmp/pip-pip-egg-info-vyczfsiy/psycopg2.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-vyczfsiy/psycopg2.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-pip-egg-info-vyczfsiy/psycopg2.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-vyczfsiy/psycopg2.egg-info/SOURCES.txt'

      Error: pg_config executable not found.

      pg_config is required to build psycopg2 from source.  Please add the directory
      containing pg_config to the $PATH or specify the full executable path with the
      option:

          python [setup.py] build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      If you prefer to avoid building psycopg2 from source, please install the PyPI
      'psycopg2-binary' package instead.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

This can be resolved in two ways:

  1. Install pg_config (in every scenario where ord-schema is used)
  2. Replace ord-schema's psycopg2 depedency with the precompiled psycopg2-binary, which is also available from PyPI.

Would you be open to option 2?

d-miketa commented 1 year ago

Embarrassingly, I see now that psycopg2's authors recommend compiling from binary every time: https://www.psycopg.org/docs/install.html#psycopg-vs-psycopg-binary

Sorry! Closing this issue.