search

EttusResearch / uhd

The USRP™ Hardware Driver Repository
http://uhd.ettus.com
Other
976 stars 655 forks source link

python import uhd fails #259

Closed citizenrich closed 5 years ago

citizenrich commented 5 years ago

Issue Description

python import uhd fails.

Setup Details

latest master into virtualenv isolated environment on macOS latest

Expected Behavior

import uhd to succeed.

Actual Behaviour

error:

$ python3
Python 3.7.2 (default, Feb 12 2019, 08:15:36)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import uhd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/__init__.py", line 10, in <module>
    from . import types
  File "/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/types.py", line 10, in <module>
    from . import libpyuhd as lib
ImportError: cannot import name 'libpyuhd' from 'uhd' (/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/__init__.py)
>>>
$ ls ~/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd
__init__.py __pycache__ filters.py  types.py    usrp.py

Steps to reproduce the problem

Followed the steps from: https://github.com/pysdr/pysdr

pip3 install virtualenv
cd $HOME
virtualenv -p python3 python_uhd_install
cd python_uhd_install
echo "export LD_LIBRARY_PATH=\$VIRTUAL_ENV/lib/" >> bin/activate
. bin/activate
pip3 install numpy mako requests six pyqt5 pyqtgraph
mkdir src && cd src
git clone https://github.com/EttusResearch/uhd.git
cd uhd/host
mkdir build && cd build
cmake -DENABLE_PYTHON3=ON -DENABLE_EXAMPLES=OFF -DENABLE_TESTS=OFF -DENABLE_RFNOC=OFF -DENABLE_C_API=OFF -DENABLE_X300=OFF -DENABLE_B100=OFF -DENABLE_USRP1=OFF -DENABLE_USRP2=OFF -DENABLE_N230=OFF -DENABLE_N300=OFF -DENABLE_E320=OFF -DENABLE_OCTOCLOCK=OFF -DENABLE_MANUAL=OFF -DCMAKE_INSTALL_PREFIX=~/python_uhd_install/ ..

# -- ######################################################
# -- # UHD enabled components
# -- ######################################################
# --   * LibUHD
# --   * LibUHD - Python API
# ...

make -j8
make install
cd $HOME/python_uhd_install
~/python_uhd_install/lib/uhd/utils/uhd_images_downloader.py
brentstapleton commented 5 years ago

Hi @citizenrich, thanks for the report! Do you see libpyuhd.so in your build directory? ie BUILD_DIR/python/uhd/libpyuhd.so or any other file named libpyuhd*?

citizenrich commented 5 years ago

Thanks for the follow-up. I've got libpyuhd.dylib in ~/python_uhd_install/src/uhd/host/build/python/uhd

(python_uhd_install) ✔ ~/python_uhd_install/src/uhd/host/build/python/uhd [master|✔]
$ ls
__init__.py filters.py  libpyuhd.dylib  types.py    usrp.py

There are a dozen or so boost warnings, which as you can see from the bash script I don't install. Thoughts? Happy to be educated here.

Edit: boost is installed via homebrew

brentstapleton commented 5 years ago

I've made an internal issue to track this. We'll get a fix as soon as we can.

I don't have a Mac set up right now, but I think I might know what the issue is, though. Could you try changing host/python/setup.py.in, line 27 to install *.dylib files?

diff --git a/host/python/setup.py.in b/host/python/setup.py.in
index 4d2df10a5..b132a9ea8 100755
--- a/host/python/setup.py.in
+++ b/host/python/setup.py.in
@@ -24,7 +24,7 @@ setup(name='uhd',
       url='https://www.ettus.com/',
       license='GPLv3',
       package_dir={'': r'${NATIVE_CURRENT_BINARY_DIR}'},
-      package_data={'uhd': ['*.so']},
+      package_data={'uhd': ['*.dylib']},
       zip_safe=False,
       packages=['uhd'],
       install_requires=['numpy'])

We use setuptools in Virtualenvs, and I think hard-coding *.so there might be causing the issue.

citizenrich commented 5 years ago

Thanks for troubleshooting. I've getting 'No module named uhd'. Here's the ls for the folder as before:

$ ls ~/python_uhd_install/src/uhd/host/build/python/uhd
__init__.py filters.py  libpyuhd.dylib  types.py    usrp.py

Edit: Doh, I had not done the make install. Error is the same:

Python 3.7.2 (default, Feb 12 2019, 08:15:36)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import uhd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/__init__.py", line 10, in <module>
    from . import types
  File "/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/types.py", line 10, in <module>
    from . import libpyuhd as lib
ImportError: cannot import name 'libpyuhd' from 'uhd' (/Users/richard/python_uhd_install/lib/python3.7/site-packages/uhd-3.15.0-py3.7.egg/uhd/__init__.py)
michaelld commented 5 years ago

Can you try this:

cd ~/python_uhd_install/src/uhd/host/build/python/uhd
ln -s libpyuhd.dylib libpyuhd.so

then try importing in Python again? Python expects the API extension to be ".so", not ".dylib". On my macOS box this change works & if it does for you then we'll fix the code internally to just use ".so" for UNIX-y systems (Windows requires a different ABI name).

michaelld commented 5 years ago

Another way to do this if you're building from source is to edit the file host/python/CMakeLists.txt and change the line that reads set(PYUHD_LIBRARY_NAME libpyuhd${CMAKE_SHARED_LIBRARY_SUFFIX}) to instead be set(PYUHD_LIBRARY_NAME libpyuhd${CMAKE_SHARED_MODULE_SUFFIX}) This works for me on my macOS box (and, I'll verify shortly on my other Mac OS X boxes, from 10.5 through 10.14, but since this is common CMake code it should work correctly regardless of the OS).

michaelld commented 5 years ago

I'm pushing this fix(MODULE instead of LIBRARY) into UHD in MacPorts. Nice and simple & I needed to update uhd-devel anyway.

citizenrich commented 5 years ago
  1. The symlink approach didn't work.
  2. I changed set(PYUHD_LIBRARY_NAME... on two lines, 45 and 91 and it works!
  3. MacPorts is cool but can I make a shameless plug for Python-enabled Homebrew as well?

Thanks for the help! Closing the issue if that's ok.

michaelld commented 5 years ago

Thanks for reporting back, @citizenrich ! Closing this ticket is fine; we'll fix it internally now that we know it's an issue. I don't use Homebrew for various reasons, and I'm really not familiar with its markup, build, and/or install structure. I find MacPorts to work quite effectively, and I like that it installs in a non-standard location by default & hence doesn't interfere with the OS system or anything else. That said, MacPorts has its issues ... & I'm sure Homebrew has its benefits.

citizenrich commented 5 years ago

FWIW, it looks like when the fix is pushed it will actually show up in homebrew.

https://github.com/Homebrew/homebrew-core/blob/master/Formula/uhd.rb#L34

citizenrich commented 5 years ago

I'm reopening this to just note that cmake fails to enable Python support in the latest release (3.14.1). It seems that this is the issue: Dependency BOOST_PYTHON_FOUND = but I searched through issues and it seems boost python has been replaced by pybind11. So, I'm confused by boost python would be a requirement. Happy to be educated on how to move forward.

Here are the steps to reproduce.

pip3 install virtualenv
cd $HOME
virtualenv -p python3 python_uhd_install
cd python_uhd_install
echo "export LD_LIBRARY_PATH=\$VIRTUAL_ENV/lib/" >> bin/activate
. bin/activate
pip3 install numpy mako requests six pyqt5 pyqtgraph pybind11
mkdir src && cd src
git clone https://github.com/EttusResearch/uhd.git
cd uhd/host
git checkout v3.14.1.0
mkdir build && cd build
cmake -DENABLE_PYTHON3=ON -DENABLE_TESTS=OFF -DENABLE_RFNOC=OFF -DENABLE_X300=OFF -DENABLE_B100=OFF -DENABLE_USRP1=OFF -DENABLE_USRP2=OFF -DENABLE_N230=OFF -DENABLE_N300=OFF -DENABLE_N320=OFF -DENABLE_E300=OFF -DENABLE_E320=OFF -DENABLE_OCTOCLOCK=OFF -DENABLE_MANUAL=OFF -DCMAKE_VERBOSE_MAKEFILE=ON ../

Abbreviated output:

-- Configuring LibUHD - Python API support...
--   Dependency ENABLE_LIBUHD = ON
--   Dependency BOOST_PYTHON_FOUND =
--   Dependency HAVE_PYTHON_MODULE_NUMPY = TRUE
--   Dependency PythonLibs_FOUND = TRUE
--   Disabling LibUHD - Python API support.
--   Override with -DENABLE_PYTHON_API=ON/OFF
...
-- ######################################################
-- # UHD enabled components
-- ######################################################
--   * LibUHD
--   * LibUHD - C API
--   * Examples
--   * Utils
--   * USB
--   * B200
--   * MPMD
--   * API/Doxygen
--   * Man Pages
--
-- ######################################################
-- # UHD disabled components
-- ######################################################
--   * LibUHD - Python API
...
mbr0wn commented 5 years ago

PyBind11 will be used for 3.15. 3.14 is still on Boost.Python.

dandymon commented 1 year ago

Lots of red herrings for some of us, for me it was that the library wasn't installed in /usr/local/lib/python3.10/dist-packages, but was in /usr/local/lib/python3.10/site-packages. I'm not an expert in the land of python and env variables, so i just copied the library to dist-packages and it works fine sudo cp /usr/local/lib/python3.10/site-packages/* /usr/local/lib/python3.10/dist-packages/ -r