search

MITHaystack / digital_rf

Read, write, and interact with data in the Digital RF and Digital Metadata formats
Other
102 stars 31 forks source link

Can't install DRF on Ubuntu 24.04 LTS #58

Open rrobinett opened 5 months ago

rrobinett commented 5 months ago

I can't find DRF in the repo for Ubuntu 24.04 LTS and pip3 installation fails:

wsprdaemon@rx888:~/wsprdaemon$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo
wsprdaemon@rx888:~/wsprdaemon$  apt-cache pkgnames | grep -i digital.*rf
wsprdaemon@rx888:~/wsprdaemon$ sudo pip3 install --break-system-packages digital_rf
Collecting digital_rf
  Using cached digital_rf-2.6.8.tar.gz (1.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [32 lines of output]
      <string>:168: SyntaxWarning: invalid escape sequence '\g'
      /tmp/pip-install-xeysqljs/digital-rf_4e1c53f7c3134998930322fdfdc9c2ba/versioneer.py:432: SyntaxWarning: invalid escape sequence '\s'
        ] = '''
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-9li51u1i/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-9li51u1i/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-9li51u1i/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-9li51u1i/overlay/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 185, in <module>
        File "/tmp/pip-install-xeysqljs/digital-rf_4e1c53f7c3134998930322fdfdc9c2ba/versioneer.py", line 1524, in get_version
          return get_versions()["version"]
                 ^^^^^^^^^^^^^^
        File "/tmp/pip-install-xeysqljs/digital-rf_4e1c53f7c3134998930322fdfdc9c2ba/versioneer.py", line 1451, in get_versions
          cfg = get_config_from_root(root)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-xeysqljs/digital-rf_4e1c53f7c3134998930322fdfdc9c2ba/versioneer.py", line 346, in get_config_from_root
          parser = configparser.SafeConfigParser()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Did you mean: 'RawConfigParser'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
wsprdaemon@rx888:~/wsprdaemon$
ryanvolz commented 5 months ago

This is happening because Ubuntu 24.04 is using Python 3.12, and the 2.6.8 release is not compatible. I just made a 2.6.9 release with the fixes for Python 3.12. You can try it now and hopefully get better results!

HB9VQQ commented 5 months ago

image

rrobinett commented 5 months ago

The installation still fails on this Ubuntu 24.04 LTS server:

@.***:~$ sudo pip3 install --break-system-packages digital_rf Collecting digital_rf Using cached digital_rf-2.6.9.tar.gz (1.2 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting h5py (from digital_rf) Using cached h5py-3.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.5 kB) Requirement already satisfied: numpy in /usr/lib/python3/dist-packages (from digital_rf) (1.26.4) Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from digital_rf) (24.0) Requirement already satisfied: python-dateutil in /usr/lib/python3/dist-packages (from digital_rf) (2.8.2) Requirement already satisfied: pytz in /usr/lib/python3/dist-packages (from digital_rf) (2024.1) Requirement already satisfied: six in /usr/lib/python3/dist-packages (from digital_rf) (1.16.0) Collecting watchdog (from digital_rf) Using cached watchdog-4.0.1-py3-none-manylinux2014_x86_64.whl.metadata (37 kB) Using cached h5py-3.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB) Using cached watchdog-4.0.1-py3-none-manylinux2014_x86_64.whl (83 kB) Building wheels for collected packages: digital_rf Building wheel for digital_rf (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for digital_rf (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [43 lines of output] Failed to load config from /tmp/pip-install-dcnjxp5b/digital-rf_6b355dddc7494ba6a84345422433ddf8/pyproject.toml: 'tool' Try to load it from setup.cfg running bdist_wheel running build running build_py Failed to load config from /tmp/pip-install-dcnjxp5b/digital-rf_6b355dddc7494ba6a84345422433ddf8/pyproject.toml: 'tool' Try to load it from setup.cfg Failed to load config from /tmp/pip-install-dcnjxp5b/digital-rf_6b355dddc7494ba6a84345422433ddf8/pyproject.toml: 'tool'

On Fri, May 24, 2024 at 2:03 AM TheOperator @.***> wrote:

image.png (view on web) https://github.com/MITHaystack/digital_rf/assets/75934980/39534482-1c04-4844-bcb5-c193e98bc781

— Reply to this email directly, view it on GitHub https://github.com/MITHaystack/digital_rf/issues/58#issuecomment-2129009723, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZBMAIGAXPDOAQJLVBDZD365PAVCNFSM6AAAAABIGJ64B6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRZGAYDSNZSGM . You are receiving this because you authored the thread.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

ryanvolz commented 5 months ago

Clearly I have more work to do with the Python build tools. Until then, you can try installing using the conda-forge package which is my primary workflow, or as a last resort installing from source.

rrobinett commented 5 months ago

Thanks, but I'll just disable the feature in my SW which requires DRF support. I have about 100 user sites and I don't want to add complexity to the installation process. Please let me know if/when I can do a 'apt install DRF..

thanks,

Rob

On Wed, May 29, 2024 at 11:41 AM Ryan Volz @.***> wrote:

Clearly I have more work to do with the Python build tools. Until then, you can try installing using the conda-forge package https://github.com/MITHaystack/digital_rf?tab=readme-ov-file#using-conda-package which is my primary workflow, or as a last resort installing from source https://github.com/MITHaystack/digital_rf?tab=readme-ov-file#using-source-code-package .

— Reply to this email directly, view it on GitHub https://github.com/MITHaystack/digital_rf/issues/58#issuecomment-2138041036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZC6KIU24ZRHKL4GKEDZEYOMJAVCNFSM6AAAAABIGJ64B6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZYGA2DCMBTGY . You are receiving this because you authored the thread.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

ryanvolz commented 5 months ago

I tried this in an Ubuntu 24.04 Docker container, and I got a similar output that was more illuminating:

Building wheels for collected packages: digital_rf
  Building wheel for digital_rf (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for digital_rf (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [43 lines of output]
      Failed to load config from /tmp/pip-install-3_hdi_z9/digital-rf_196dd17ba4c54cd5b0bc8f5e3ba3ada0/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      running bdist_wheel
      running build
      running build_py
      Failed to load config from /tmp/pip-install-3_hdi_z9/digital-rf_196dd17ba4c54cd5b0bc8f5e3ba3ada0/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      Failed to load config from /tmp/pip-install-3_hdi_z9/digital-rf_196dd17ba4c54cd5b0bc8f5e3ba3ada0/pyproject.toml: 'tool'
      Try to load it from setup.cfg
      creating build
      creating build/lib.linux-x86_64-cpython-312
      creating build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/util.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/ringbuffer.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/digital_rf_deprecated_hdf5.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/list_drf.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/__init__.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/drf_command.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/_version.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/watchdog_drf.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/digital_metadata.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/mirror.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      copying digital_rf/digital_rf_hdf5.py -> build/lib.linux-x86_64-cpython-312/digital_rf
      creating build/lib.linux-x86_64-cpython-312/gr_digital_rf
      copying gr_digital_rf/vector.py -> build/lib.linux-x86_64-cpython-312/gr_digital_rf
      copying gr_digital_rf/__init__.py -> build/lib.linux-x86_64-cpython-312/gr_digital_rf
      copying gr_digital_rf/digital_rf_sink.py -> build/lib.linux-x86_64-cpython-312/gr_digital_rf
      copying gr_digital_rf/digital_rf_source.py -> build/lib.linux-x86_64-cpython-312/gr_digital_rf
      copying gr_digital_rf/raster.py -> build/lib.linux-x86_64-cpython-312/gr_digital_rf
      UPDATING build/lib.linux-x86_64-cpython-312/digital_rf/_version.py
      set build/lib.linux-x86_64-cpython-312/digital_rf/_version.py to '2.6.9'
      running build_ext
      INFO: pkg-config not installed. Defaulting to HDF5_ROOT="/usr"
      building 'digital_rf._py_rf_write_hdf5' extension
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/lib
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/tmp/pip-install-3_hdi_z9/digital-rf_196dd17ba4c54cd5b0bc8f5e3ba3ada0/include -I/usr/include/python3.12 -I/tmp/pip-build-env-c8h42f18/overlay/local/lib/python3.12/dist-packages/numpy/core/include -I/usr/include -I/opt/local/include -I/usr/local/include -c lib/py_rf_write_hdf5.c -o build/temp.linux-x86_64-cpython-312/lib/py_rf_write_hdf5.o
      In file included from lib/py_rf_write_hdf5.c:24:
      /tmp/pip-install-3_hdi_z9/digital-rf_196dd17ba4c54cd5b0bc8f5e3ba3ada0/include/digital_rf.h:46:10: fatal error: hdf5.h: No such file or directory
         46 | #include "hdf5.h"
            |          ^~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for digital_rf
Failed to build digital_rf
ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects

I then had to apt install libhdf5-dev pkg-config to have all of the necessary dependencies to complete the build. Can you ensure these non-Python external dependencies before running pip3 install digital_rf?

HB9VQQ commented 5 months ago

wsprdaemon@rx888:~/wsprdaemon$ sudo pip3 install digital_rf error: externally-managed-environment

× This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification.

ryanvolz commented 5 months ago

wsprdaemon@rx888:~/wsprdaemon$ sudo pip3 install digital_rf error: externally-managed-environment

I think you need to add --break-system-packages as an override like it suggests, since I think the intention with wsprdaemon is indeed to install into the system Python environment instead of some sort of virtual environment. That's a choice that falls outside of Digital RF. Sorry for the confusion since I dropped that flag in my previous comment!

rrobinett commented 5 months ago

HI Ryan,

Thanks for your attention. I think I tried -break-system-packages, but I'll try it again later today.

Rob

On Tue, Jun 4, 2024 at 8:51 AM Ryan Volz @.***> wrote:

@.***:~/wsprdaemon$ sudo pip3 install digital_rf error: externally-managed-environment

I think you need to add --break-system-packages as an override like it suggests, since I think the intention with wsprdaemon is indeed to install into the system Python environment instead of some sort of virtual environment. That's a choice that falls outside of Digital RF. Sorry for the confusion since I dropped that flag in my previous comment!

— Reply to this email directly, view it on GitHub https://github.com/MITHaystack/digital_rf/issues/58#issuecomment-2147872924, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIFAQZHH4NANS3MFVT4QH6TZFXO6RAVCNFSM6AAAAABIGJ64B6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBXHA3TEOJSGQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

HB9VQQ commented 5 months ago

--break-system-packages sounds pretty scary, I'm not sure I'm brave enough to do that. I'll wait for Rob :-)

rrobinett commented 5 months ago

I have added two of my collaborators to this thread. While implementing support for Debian 12 and Ubuntu 24.04 I have added several "--break-system-packages" in order to execute some of the Python packages used in my Wsprdaemon software. I think I tried "--break-system-packages" and got an internal error in the python program, but I've got a dozen debug projects and I don't remember the details....

rrobinett commented 5 months ago

Hi Ryan,

I just tried to install DRF on an Ubuntu 24.04 system with ' -break-system-packages' and got this error

Rob

...... creating build/temp.linux-x86_64-cpython-312/lib x8664-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/tmp/pip-install-34ubl04/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include -I/usr/include/python3.12 -I/tmp/pip-build-env-5qjn0la4/overlay/local/lib/python3.12/dist-packages/numpy/core/include -I/usr/include -I/opt/local/include -I/usr/local/include -c lib/py_rf_write_hdf5.c -o build/temp.linux-x86_64-cpython-312/lib/py_rf_write_hdf5.o In file included from lib/py_rf_write_hdf5.c:24:

/tmp/pip-install-34ubl04_/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include/digital_rf.h:46:10: fatal error: hdf5.h: No such file or directory 46 | #include "hdf5.h" | ^~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for digital_rf Failed to build digital_rf ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects

On Wed, Jun 5, 2024 at 2:26 PM Rob Robinett @.***> wrote:

I have added two of my collaborators to this thread. While implementing support for Debian 12 and Ubuntu 24.04 I have added several "--break-system-packages" in order to execute some of the Python packages used in my Wsprdaemon software. I think I tried "--break-system-packages" and got an internal error in the python program, but I've got a dozen debug projects and I don't remember the details....

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

rrobinett commented 5 months ago

Install libhdf5-dev with apt first.

On Jun 8, 2024, at 08:34, Rob Robinett @.***> wrote:

Hi Ryan,

I just tried to install DRF on an Ubuntu 24.04 system with ' -break-system-packages' and got this error

Rob

...... creating build/temp.linux-x86_64-cpython-312/lib x8664-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/tmp/pip-install-34ubl04/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include -I/usr/include/python3.12 -I/tmp/pip-build-env-5qjn0la4/overlay/local/lib/python3.12/dist-packages/numpy/core/include -I/usr/include -I/opt/local/include -I/usr/local/include -c lib/py_rf_write_hdf5.c -o build/temp.linux-x86_64-cpython-312/lib/py_rf_write_hdf5.o In file included from lib/py_rf_writehdf5.c:24: /tmp/pip-install-34ubl04/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include/digital_rf.h:46:10: fatal error: hdf5.h: No such file or directory 46 | #include "hdf5.h" | ^~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for digital_rf Failed to build digital_rf ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects

On Wed, Jun 5, 2024 at 2:26 PM Rob Robinett @. @.>> wrote:

I have added two of my collaborators to this thread. While implementing support for Debian 12 and Ubuntu 24.04 I have added several "--break-system-packages" in order to execute some of the Python packages used in my Wsprdaemon software. I think I tried "--break-system-packages" and got an internal error in the python program, but I've got a dozen debug projects and I don't remember the details....

-- Rob Robinett AI6VN @. @.> mobile: +1 650 218 8896

rrobinett commented 5 months ago

Thank you Michael and Ryan, installation works on 24.04. So to summarize, on DRF can be installed and run on Ubuntu 24.04 with the addition of ' -break-system-packages' on the pip3 command line and 'apt install libhdf5-dev'

On Sat, Jun 8, 2024 at 6:37 AM Michael James Hauan @.***> wrote:

Install libhdf5-dev with apt first.

On Jun 8, 2024, at 08:34, Rob Robinett @.***> wrote:

Hi Ryan,

I just tried to install DRF on an Ubuntu 24.04 system with ' -break-system-packages' and got this error

Rob

...... creating build/temp.linux-x86_64-cpython-312/lib x8664-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -fPIC -I/tmp/pip-install-34ubl04/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include -I/usr/include/python3.12 -I/tmp/pip-build-env-5qjn0la4/overlay/local/lib/python3.12/dist-packages/numpy/core/include -I/usr/include -I/opt/local/include -I/usr/local/include -c lib/py_rf_write_hdf5.c -o build/temp.linux-x86_64-cpython-312/lib/py_rf_write_hdf5.o In file included from lib/py_rf_write_hdf5.c:24:

/tmp/pip-install-34ubl04_/digital-rf_0446170ae2a743639aad2bdf4bdc4ef4/include/digital_rf.h:46:10: fatal error: hdf5.h: No such file or directory 46 | #include "hdf5.h" | ^~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for digital_rf Failed to build digital_rf ERROR: Could not build wheels for digital_rf, which is required to install pyproject.toml-based projects

On Wed, Jun 5, 2024 at 2:26 PM Rob Robinett @.***> wrote:

I have added two of my collaborators to this thread. While implementing support for Debian 12 and Ubuntu 24.04 I have added several "--break-system-packages" in order to execute some of the Python packages used in my Wsprdaemon software. I think I tried "--break-system-packages" and got an internal error in the python program, but I've got a dozen debug projects and I don't remember the details....

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896

-- Rob Robinett AI6VN @.*** mobile: +1 650 218 8896