search

gnuradio / pybombs

PyBOMBS (Python Build Overlay Managed Bundle System) is the GNU Radio install management system for resolving dependencies and pulling in out-of-tree projects.
https://gnuradio.org
GNU General Public License v3.0
414 stars 189 forks source link

pybombs fails to install gnuradio on python2/3 system #553

Open Sec42 opened 4 years ago

Sec42 commented 4 years ago

After following the documented steps of

pybombs recipes add-defaults
pybombs prefix init -a default ~/gnuradio -R gnuradio-default
cd ~/gnuradio
source setup_env.sh

the resulting gnuradio installation does not work:

sec@fukuzatsu:~/gnuradio$ gr_plot
Traceback (most recent call last):
  File "/home/sec/gnuradio/bin/gr_plot", line 25, in <module>
    from gnuradio.plot_data import plot_data
ModuleNotFoundError: No module named 'gnuradio'

No indication of warnings or errors is given during installation:

Logs:

fukuzatsu:~/Projects/PyBOMBS>pybombs recipes add-defaults
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.16
PyBOMBS - INFO - PyBOMBS Version 2.3.3
Cloning into 'gr-etcetera'...
remote: Enumerating objects: 72, done.
remote: Counting objects: 100% (72/72), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 465 (delta 46), reused 35 (delta 19), pack-reused 393
Receiving objects: 100% (465/465), 90.93 KiB | 716.00 KiB/s, done.
Resolving deltas: 100% (309/309), done.
Cloning into 'gr-recipes'...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 1752 (delta 14), reused 14 (delta 4), pack-reused 1727
Receiving objects: 100% (1752/1752), 358.19 KiB | 1.40 MiB/s, done.
Resolving deltas: 100% (1434/1434), done.
fukuzatsu:~/Projects/PyBOMBS>pybombs prefix init -a default ~/gnuradio -R gnuradio-default
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.16
PyBOMBS - INFO - PyBOMBS Version 2.3.3
PyBOMBS.prefix - INFO - Creating directory `/home/sec/gnuradio'
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.16
PyBOMBS.ConfigManager - INFO - Creating new config file /home/sec/gnuradio/.pybombs/config.yml
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.16
PyBOMBS.prefix - INFO - Installing default packages for prefix...
PyBOMBS.prefix - INFO - 
  - gnuradio
PyBOMBS.install_manager - INFO - Phase 1: Creating install tree and installing binary packages:
Install tree:
|
\- gnuradio
   |
   \- uhd
PyBOMBS.install_manager - INFO - Phase 2: Recursively installing source package to prefix:
PyBOMBS.install_manager - INFO - Installing package: uhd
Cloning into 'uhd'...
remote: Enumerating objects: 2557, done.
remote: Counting objects: 100% (2557/2557), done.
remote: Compressing objects: 100% (868/868), done.
remote: Total 87230 (delta 1797), reused 2243 (delta 1685), pack-reused 84673
Receiving objects: 100% (87230/87230), 115.72 MiB | 5.32 MiB/s, done.
Resolving deltas: 100% (68012/68012), done.
Configuring: (100%) [=========================================================]
Building:    (100%) [=========================================================]
Installing:  (100%) [=========================================================]
PyBOMBS.install_manager - INFO - Installation successful.
PyBOMBS.install_manager - INFO - Installing package: gnuradio
Cloning into 'gnuradio'...
remote: Enumerating objects: 147178, done.
remote: Total 147178 (delta 0), reused 0 (delta 0), pack-reused 147178
Receiving objects: 100% (147178/147178), 52.20 MiB | 1002.00 KiB/s, done.
Resolving deltas: 100% (114327/114327), done.
Submodule 'volk' (https://github.com/gnuradio/volk.git) registered for path 'volk'
Cloning into '/home/sec/gnuradio/src/gnuradio/volk'...
remote: Enumerating objects: 7961, done.        
remote: Total 7961 (delta 0), reused 0 (delta 0), pack-reused 7961
Receiving objects: 100% (7961/7961), 2.16 MiB | 3.45 MiB/s, done.
Resolving deltas: 100% (5531/5531), done.
Submodule path 'volk': checked out '1299d72c396a88fd2679adfd7a919ac00d2cf678'
Configuring: (100%) [=========================================================]
Building:    (100%) [=========================================================]
Installing:  (100%) [=========================================================]
PyBOMBS.install_manager - INFO - Installation successful.

Versions:

fukuzatsu:~/Projects/PyBOMBS>pybombs inv    
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.16
PyBOMBS - INFO - PyBOMBS Version 2.3.3
Showing package state:
uhd:                installed
gnuradio:           installed
fukuzatsu:~/Projects/PyBOMBS>cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=19.04
DISTRIB_CODENAME=disco
DISTRIB_DESCRIPTION="Ubuntu 19.04"
fukuzatsu:~>python -V
Python 2.7.16
fukuzatsu:~>python2 -V
Python 2.7.16
fukuzatsu:~>python3 -V
Python 3.7.3

As far as I can tell, this is due to gnuradio building against python3 whereas pybombs assumes it built for python2.

fukuzatsu:~/gnuradio>grep PYT setup_env.sh
export PYTHONPATH="/home/sec/gnuradio/python:/home/sec/gnuradio/lib/python2.6/site-packages:/home/sec/gnuradio/lib64/python2.6/site-packages:/home/sec/gnuradio/lib/python2.6/dist-packages:/home/sec/gnuradio/lib64/python2.6/dist-packages:/home/sec/gnuradio/lib/python2.7/site-packages:/home/sec/gnuradio/lib64/python2.7/site-packages:/home/sec/gnuradio/lib/python2.7/dist-packages:/home/sec/gnuradio/lib64/python2.7/dist-packages:$PYTHONPATH"

and

fukuzatsu:~/gnuradio>find . -wholename \*/gnuradio/__init__.py
./src/gnuradio/gnuradio-runtime/python/gnuradio/__init__.py
./lib/python3.7/dist-packages/gnuradio/__init__.py
cclamb commented 4 years ago

I'm not sure if I'm being pedantic here, but the fix is just to insert the new python libraries.

After a successful 3.8 install using the command sequence:

sudo apt-get install python-pip sudo pip install pybombs pybombs auto-config pybombs recipes add-defaults pybombs prefix init ~/gnuradio -R gnuradio-default (wait a while) . ~/gnuradio/setup_env.sh gnuradio-companion

gnu radio-companion will not run. The python path in the setup_env.sh does not reference the python 3 and python 3.7 site-packages directories. To fix, add the correct python path information including the new python 3 and 3.7 site-packages directories to the PYTHONPATH environment variable e.g.:

export PYTHONPATH="/home/cclamb/gnuradio/lib/python3/dist-packages:/home/cclamb/gnuradio/lib/python3.7/dist-packages:/home/cclamb/gnuradio/python:/home/cclamb/gnuradio/lib/python2.6/site-packages:/home/cclamb/gnuradio/lib64/python2.6/site-packages:/home/cclamb/gnuradio/ lib/python2.6/dist-packages:/home/cclamb/gnuradio/lib64/python2.6/dist-packages:/home/cclamb/gnuradio/lib/python2.7/site-packages:/home/cclamb/gnuradio/lib64/python2.7/site-packages:/home/cclamb/gnuradio/lib/python2.7/dist-packages:/home/cclamb/gnuradio/lib64/python2.7/dist-packages:$PYTHONPATH"

ArtemPisarenko commented 4 years ago

Same problem on Ubuntu 18.04.2 LTS

Sec42 commented 4 years ago

This is still broken on Ubuntu 19.10. The issue that pybombs creates a broken PYTHONPATH only listing python2 paths (which do not exist at all) remains.

A quick edit to setup_env.sh removing all python2 references, and replacing them with python3 seems to make the installation work.

schneider42 commented 4 years ago

On my "Ubuntu 18.04.3 LTS" the following happens when installing 3.8 for Python 3 via PyBOMBS:

leading to Python not finding gnuradio.

Commands executed:

sudo pip3 install --upgrade git+https://github.com/gnuradio/pybombs.git
pybombs auto-config
pybombs recipes add-defaults
pybombs prefix init ~/gr-3.8 -R gnuradio-default
phil61 commented 4 years ago

I've observed the same behaviour as schneider42.

It appears that pybombs is incorrectly generating the python paths to dist-packages.

I'm successfully working around this issue by manually editing the setup_env.sh to change references to 'python3.6/dist-packages' into 'python3/dist-packages'.

gnuradio-companion works fine after the edit and I no longer receive an error when I import gnuradio into my python scripts.

gareth8118 commented 4 years ago

I have also found it broken on Ubuntu 19.10 after using the Quick Start on a fresh OS install.

The specific edit I needed to get gnuradio-companion to run is:

--- setup_env.sh.orig   2020-02-21 20:44:00.694360807 +0000
+++ setup_env.sh    2020-02-21 20:46:00.295175376 +0000
@@ -1,6 +1,6 @@
 # WARNING: This file is auto-generated by pybombs, any manual changes to it may be overwritten!
 export PATH="/home/gareth/gr38/bin:$PATH"
-export PYTHONPATH="/home/gareth/gr38/lib/python3.7/site-packages:/home/gareth/gr38/lib/python3.7/dist-packages:/home/gareth/gr38/lib64/python3.7/site-packages:/home/gareth/gr38/lib64/python3.7/dist-packages:$PYTHONPATH"
+export PYTHONPATH="/home/gareth/gr38/lib/python3/dist-packages:/home/gareth/gr38/lib/python3.7/site-packages:/home/gareth/gr38/lib/python3.7/dist-packages:/home/gareth/gr38/lib64/python3.7/site-packages:/home/gareth/gr38/lib64/python3.7/dist-packages:$PYTHONPATH"
 export LD_LIBRARY_PATH="/home/gareth/gr38/lib:/home/gareth/gr38/lib64/:$LD_LIBRARY_PATH"
 export LIBRARY_PATH="/home/gareth/gr38/lib:/home/gareth/gr38/lib64/:$LIBRARY_PATH"
 export PKG_CONFIG_PATH="/home/gareth/gr38/lib/pkgconfig:/home/gareth/gr38/lib64/pkgconfig:$PKG_CONFIG_PATH"
wgaylord commented 4 years ago

Still occurring.

Jim2435074 commented 4 years ago

gareth8118 seems to have a correct fix. site-packages and dist-packages are installed in separate python3 and python3.x directories. It is a quick fix on Ubuntu 18.04. Note this was NOT an issue when using python2 and gnuradio-stable.

drws commented 4 years ago

Still occurring on Ubuntu 18.04.5 with PyBOMBS 2.3.4. As the default is Python 2.7, the generated PYTHONPATH consisted of (all non-existing!) .../python2*/... paths, even though GR 3.8 uses Python 3. I was also able to fix it with Gareth's edit, including Python 3.6 path correction (Ubuntu 18.04).

delcencen commented 3 years ago

i ave the solution...do first: sudo apt-get install python3-setuptools

drws commented 3 years ago

In my case, python3-setuptools were installed as well as python-setuptools (python2-setuptools). There is probably something wrong with the PYTHONPATH derivation logic in PyBOMBS.

Sec42 commented 3 years ago

As a workaround, run the following snipped after pybombs prefix init inside you new prefix:

( # Fix pybombs paths
python=python3
maj=$($python -V 2>&1 |tr . ' '|cut -d ' ' -f 2)
min=$($python -V 2>&1 |tr . ' '|cut -d ' ' -f 3)
echo -n "export PYTHONPATH="
for ver in $maj $maj.$min ; do
for lib in lib lib64 ; do
for pkg in site dist ; do
echo -n $(pwd)/${lib}/python${ver}/${pkg}-packages:
done
done
done
echo '${PYTHONPATH}'
) >>setup_env.sh

It grabs the correct version number from your python binary and adds the proper paths.