cfitsio build failure

[blanton144]

On a Linux Ubuntu system (33~22.04.1-Ubuntu) in Python 3.11, I am encountering an error with, 'pip install fitsio', which is trying to install fitsio 1.2.0. The error is below.

Collecting fitsio
  Using cached fitsio-1.2.0.tar.gz (4.6 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in ./venv/test-lincc-release/lib/python3.11/site-packages (from fitsio) (1.26.0)
Building wheels for collected packages: fitsio
  Building wheel for fitsio (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for fitsio (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [191 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/fitsio
      copying fitsio/__init__.py -> build/lib.linux-x86_64-cpython-311/fitsio
      copying fitsio/util.py -> build/lib.linux-x86_64-cpython-311/fitsio
      copying fitsio/fitslib.py -> build/lib.linux-x86_64-cpython-311/fitsio
      copying fitsio/header.py -> build/lib.linux-x86_64-cpython-311/fitsio
      creating build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/makedata.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_warnings.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_table.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_header_junk.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_image_compression.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_lib.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/checks.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_header.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      copying fitsio/tests/test_image.py -> build/lib.linux-x86_64-cpython-311/fitsio/tests
      creating build/lib.linux-x86_64-cpython-311/fitsio/hdu
      copying fitsio/hdu/__init__.py -> build/lib.linux-x86_64-cpython-311/fitsio/hdu
      copying fitsio/hdu/table.py -> build/lib.linux-x86_64-cpython-311/fitsio/hdu
      copying fitsio/hdu/base.py -> build/lib.linux-x86_64-cpython-311/fitsio/hdu
      copying fitsio/hdu/image.py -> build/lib.linux-x86_64-cpython-311/fitsio/hdu
      running egg_info
      writing fitsio.egg-info/PKG-INFO
      writing dependency_links to fitsio.egg-info/dependency_links.txt
      writing requirements to fitsio.egg-info/requires.txt
      writing top-level names to fitsio.egg-info/top_level.txt
      reading manifest file 'fitsio.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE.txt'
      writing manifest file 'fitsio.egg-info/SOURCES.txt'
      /tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'fitsio.test_images' is absent from the `packages` configuration.
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'fitsio.test_images' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'fitsio.test_images' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'fitsio.test_images' to be distributed and are
              already explicitly excluding 'fitsio.test_images' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html

              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      copying fitsio/fitsio_pywrap.c -> build/lib.linux-x86_64-cpython-311/fitsio
      creating build/lib.linux-x86_64-cpython-311/fitsio/test_images
      copying fitsio/test_images/test_gzip_compressed_image.fits.fz -> build/lib.linux-x86_64-cpython-311/fitsio/test_images
      running build_ext
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/fitscore.c
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/fitscore.c
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/putcols.c
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/putcols.c
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/fitsio.h
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/fitsio.h
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/configure.in
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/configure.in
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/Makefile.in
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/Makefile.in
      checking file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/configure
      patching file build/temp.linux-x86_64-cpython-311/cfitsio-4.2.0/configure
      checking build system type... x86_64-pc-linux-gnu
      checking host system type... x86_64-pc-linux-gnu
      checking target system type... x86_64-pc-linux-gnu
      checking for gcc... gcc
      checking whether the C compiler works... yes
      checking for C compiler default output file name... a.out
      checking for suffix of executables...
      checking whether we are cross compiling... no
      checking for suffix of object files... o
      checking whether the compiler supports GNU C... yes
      checking whether gcc accepts -g... yes
      checking for gcc option to enable C11 features... none needed
      checking for stdio.h... yes
      checking for stdlib.h... yes
      checking for string.h... yes
      checking for inttypes.h... yes
      checking for stdint.h... yes
      checking for strings.h... yes
      checking for sys/stat.h... yes
      checking for sys/types.h... yes
      checking for unistd.h... yes
      checking for main in -lbz2... yes
      checking for bzlib.h... yes
      checking for gcc... (cached) gcc
      checking whether the compiler supports GNU C... (cached) yes
      checking whether gcc accepts -g... (cached) yes
      checking for gcc option to enable C11 features... (cached) none needed
      checking for gfortran... gfortran
      checking whether we are using GNU Fortran...
      yes
      cfitsio: == Adding wrapper support for GNU Fortran...
       done
      checking for ar... ar
      checking for ranlib... ranlib
      checking for stdlib.h... (cached) yes
      checking for string.h... (cached) yes
      checking for math.h... yes
      checking for limits.h... yes
      cfitsio: == Using gcc version 11
      checking for curl-config... curl-config
      checking for inflateEnd in -lz... no
      configure: error: Unable to locate zlib compression library
      Traceback (most recent call last):
        File "/home/blanton/venv/test-lincc-release/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/blanton/venv/test-lincc-release/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/blanton/venv/test-lincc-release/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 434, in build_wheel
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 507, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 313, in <module>
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "<string>", line 106, in run
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-scg3c2db/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "<string>", line 130, in build_extensions
        File "<string>", line 266, in configure_cfitsio
      ValueError: could not configure cfitsio 4.2.0
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for fitsio
Failed to build fitsio
ERROR: Could not build wheels for fitsio, which is required to install pyproject.toml-based projects

[beckermr]

Looks like the build cannot find zlib. What kinds of variables that effect include and linking are in your environment?

[blanton144]

Like LD_LIBRARY_PATH or C_INCLUDE_PATH? Nothing like that (I think!). My env is reproduced below ...

SYSTEMD_EXEC_PID=2044
MODULEPATH=/home/blanton/modulefiles:/usr/modulefiles/Linux:/usr/modulefiles/Core:/usr/share/lmod/lmod/modulefiles/Core
VENDOR=unknown
SESSION_MANAGER=local/snow:@/tmp/.ICE-unix/2000,unix/snow:/tmp/.ICE-unix/2000
GNOME_TERMINAL_SCREEN=/org/gnome/Terminal/screen/d491ca30_07bc_4e5a_b46c_b6e8407f7112
XDG_CURRENT_DESKTOP=ubuntu:GNOME
LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod
WAYLAND_DISPLAY=wayland-0
TARG_TITLE_BAR_PAREN= 
LMOD_DIR=/usr/share/lmod/lmod/libexec
LMOD_SETTARG_CMD=:
SSH_AUTH_SOCK=/tmp/ssh-XXXXXXRusQuP/agent.94762
COLORTERM=truecolor
QT_IM_MODULE=ibus
USER=blanton
DESKTOP_SESSION=ubuntu
XDG_MENU_PREFIX=gnome-
HOME=/home/blanton
BASH_ENV=/usr/share/lmod/lmod/init/bash
MODULESHOME=/usr/share/lmod/lmod
LMOD_COLORIZE=yes
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
MANPATH=/usr/share/lmod/lmod/share/man::
MODULEPATH_ROOT=/usr/modulefiles
LMOD_sys=Linux
SSH_AGENT_LAUNCHER=gnome-keyring
GTK_MODULES=gail:atk-bridge
XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/var/lib/snapd/desktop
OSTYPE=linux
XDG_SESSION_DESKTOP=ubuntu
MACHTYPE=x86_64
QT_ACCESSIBILITY=1
LMOD_PKG=/usr/share/lmod/lmod
LMOD_PREPEND_BLOCK=normal
GNOME_SETUP_DISPLAY=:1
LANG=en_US.UTF-8
LOGNAME=blanton
GNOME_TERMINAL_SERVICE=:1.124
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
HOST=snow
GNOME_SHELL_SESSION_MODE=ubuntu
LMOD_FULL_SETTARG_SUPPORT=no
PATH=/home/blanton/venv/test-lincc-release/bin:/opt/cisco/anyconnect/bin:/home/blanton/anaconda3/bin:/usr/local/bin:/home/blanton/.local/bin:/home/blanton/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
XDG_RUNTIME_DIR=/run/user/1000
XDG_SESSION_TYPE=wayland
LMOD_arch=x86_64
HOSTTYPE=x86_64-linux
LMOD_VERSION=6.6
SHELL=/bin/tcsh
VTE_VERSION=6800
USERNAME=blanton
XMODIFIERS=@im=ibus
XAUTHORITY=/run/user/1000/.mutter-Xwaylandauth.VN8OB2
PWD=/home/blanton
GROUP=blanton
XDG_SESSION_CLASS=user
TERM=wsvt25m
GDMSESSION=ubuntu
DISPLAY=:0
SHLVL=1
PAGER=less
EDITOR=vi
RSYNC_RSH=ssh
PERLLIB=/home/blanton/Dropbox/unixenv/perl
CONDA_EXE=/home/blanton/anaconda3/bin/conda
_CONDA_ROOT=/home/blanton/anaconda3
_CONDA_EXE=/home/blanton/anaconda3/bin/conda
CONDA_PYTHON_EXE=/home/blanton/anaconda3/bin/python
CONDA_SHLVL=0
BROWSER=/snap/bin/firefox
SSH_AGENT_PID=94763
VIRTUAL_ENV=/home/blanton/venv/test-lincc-release
VIRTUAL_ENV_PROMPT=(test-lincc-release) 

[beckermr]

Ok. Do you have zlib locally somewhere?

[blanton144]

"apt install zlib1g" claims it is already installed, yes. Presumably wherever Ubuntu normally puts it. I am not 100% sure what it calls itself, but there is this:

/usr/lib/x86_64-linux-gnu/libz.so.1

A bit surprised configure can't find that, but I haven't really used configure in anger in a very long time!

[beckermr]

Ok. It looks like you have a conda env going? You can use conda install fitsio maybe?

[beckermr]

You'll need conda-forge in your channels.

[blanton144]

For using fitsio myself yes that does work. And in pretty much every context I work we have miniconda or conda distributions.

My test was of installing the dependencies of a different product (kcorrect) in a generic Python distribution on Ubuntu (in this case a venv). That failed, and I would prefer it not fail, so I removed the fitsio dependency in favor of astropy.io.fits (the dependency on FITS I/O is really only in two lines in the code).

But it might still be considered a problem if 'pip install fitsio==1.2.0' fails on non-Anaconda Python distributions ... I will leave it up to you to decide that!

[esheldon]

Mike, can you please post a minimal example of using the ubuntu python to create a venv and install fitsio?