Crash when using Spectrum Analyzer with GR backend

ZeroChaos- commented 5 years ago

Expected Behavior

I expect this to work

Actual Behavior

wild errors when urh does silly things. Ideally, on a python3 installation I shouldn't see things like /usr/bin/env python2

Steps To Reproduce

zero@theprophet urh % grep python2 $(qlist urh) (git)-[master] /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: if self.backend_handler.python2_exe: /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.ui.lineEditPython2Interpreter.setText(self.backend_handler.python2_exe) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.ui.lineEditPython2Interpreter.editingFinished.connect(self.on_python2_exe_path_edited) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.ui.btnChoosePython2Interpreter.clicked.connect(self.on_btn_choose_python2_interpreter_clicked) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.ui.radioButtonPython2Interpreter.clicked.connect(self.on_radio_button_python2_interpreter_clicked) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.ui.lineEditPython2Interpreter.setText(self.backend_handler.python2_exe) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.tr("Gnuradio is not installed or incompatible with python2 interpreter.")) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: self.backend_handler.python2_exe = self.ui.lineEditPython2Interpreter.text() /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: def on_python2_exe_path_edited(self): /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: def on_btn_choose_python2_interpreterclicked(self): /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: filename, = QFileDialog.getOpenFileName(self, self.tr("Choose python2 interpreter"), filter=dialog_filter) /usr/lib64/python3.6/site-packages/urh/controller/dialogs/OptionsDialog.py: def on_radio_button_python2_interpreter_clicked(self, checked: bool): Binary file /usr/lib64/python3.6/site-packages/urh/controller/dialogs/pycache/OptionsDialog.cpython-36.opt-2.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/controller/dialogs/pycache/OptionsDialog.cpython-36.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/controller/dialogs/pycache/OptionsDialog.cpython-36.opt-1.pyc matches /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/airspy_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/bladerf_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/bladerf_send.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/funcube_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/hackrf_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/hackrf_send.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/rtl-sdr_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/sdrplay_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/usrp_recv.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/usrp_send.py:#!/usr/bin/env python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: if os.path.isfile(os.path.join(gnuradio_dir, "gr-python27", "pythonw.exe")): /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: self.python2_interpreter = os.path.join(gnuradio_dir, "gr-python27", "pythonw.exe") /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: self.python2_interpreter = os.path.join(gnuradio_dir, "gr-python27", "python.exe") /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: self.python2_interpreter = constants.SETTINGS.value("python2_exe", "") /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: if not self.python2_interpreter: /usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py: options = [self.python2_interpreter, os.path.join(rp, filename), Binary file /usr/lib64/python3.6/site-packages/urh/dev/gr/pycache/AbstractBaseThread.cpython-36.opt-2.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/dev/gr/pycache/AbstractBaseThread.cpython-36.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/dev/gr/pycache/AbstractBaseThread.cpython-36.opt-1.pyc matches /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: python2_exe = constants.SETTINGS.value('python2_exe', '') /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: if not python2_exe: /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: self.python2_exe = self.__get_python2_interpreter() /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: constants.SETTINGS.setValue("python2_exe", self.python2_exe) /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: self.python2_exe = python2_exe /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: constants.SETTINGS.setValue("python2_exe", self.python2_exe) /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: def python2_exe(self): /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: return self.python2_exe /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: @python2_exe.setter /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: def python2_exe(self, value): /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: if value != self.python2_exe: /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: self.python2_exe = value /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: constants.SETTINGS.setValue("python2_exe", value) /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: if os.path.isfile(self.python2_exe) and os.access(self.python2_exe, os.X_OK): /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: self.gnuradio_is_installed = call('"{0}" -c "import gnuradio"'.format(self.python2_exe), /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: def get_python2_interpreter(self): /usr/lib64/python3.6/site-packages/urh/dev/BackendHandler.py: for prog in ["python2", "python2.exe"]: Binary file /usr/lib64/python3.6/site-packages/urh/dev/pycache/BackendHandler.cpython-36.opt-2.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/dev/pycache/BackendHandler.cpython-36.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/dev/pycache/BackendHandler.cpython-36.opt-1.pyc matches /usr/lib64/python3.6/site-packages/urh/ui/ui_options.py: self.lineEditPython2Interpreter.setPlaceholderText(_translate("DialogOptions", "/usr/bin/python2")) Binary file /usr/lib64/python3.6/site-packages/urh/ui/pycache/ui_options.cpython-36.opt-2.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/ui/pycache/ui_options.cpython-36.pyc matches Binary file /usr/lib64/python3.6/site-packages/urh/ui/pycache/ui_options.cpython-36.opt-1.pyc matches

Platform Specifications

OS: Gentoo Linux
URH version: 2.5.2
Python version: 3.6
Installed via emerge (I am the maintainer of the gentoo package)

jopohl commented 5 years ago

The python2 stuff is required for the GNU radio backend, which is optional. So what is the exact problem? Especially, please specify this line:

wild errors when urh does silly things

blshkv commented 5 years ago

Our user reported the following:

pentoo-usb ~ # urh
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Skipping native support for airspy
Found bladeRF lib. Will compile with native bladerf support
    Detected BLADERF_API_VERSION v2.02
Found hackrf lib. Will compile with native hackrf support
Skipping native support for limesdr
Found iio lib. Will compile with native plutosdr support
Found rtlsdr lib. Will compile with native rtlsdr support
Found uhd lib. Will compile with native usrp support
Skipping native support for sdrplay
Skipping native support for airspy
Found bladeRF lib. Will compile with native bladerf support
    Detected BLADERF_API_VERSION v2.02
Found hackrf lib. Will compile with native hackrf support
Skipping native support for limesdr
Found iio lib. Will compile with native plutosdr support
Found rtlsdr lib. Will compile with native rtlsdr support
Found uhd lib. Will compile with native usrp support
Skipping native support for sdrplay
Compiling /usr/lib64/python3.6/site-packages/urh/dev/native/lib/bladerf.pyx because it changed.
Compiling /usr/lib64/python3.6/site-packages/urh/dev/native/lib/hackrf.pyx because it changed.
Compiling /usr/lib64/python3.6/site-packages/urh/dev/native/lib/plutosdr.pyx because it changed.
Compiling /usr/lib64/python3.6/site-packages/urh/dev/native/lib/rtlsdr.pyx because it changed.
Compiling /usr/lib64/python3.6/site-packages/urh/dev/native/lib/usrp.pyx because it changed.
[1/5] Cythonizing /usr/lib64/python3.6/site-packages/urh/dev/native/lib/bladerf.pyx
[2/5] Cythonizing /usr/lib64/python3.6/site-packages/urh/dev/native/lib/hackrf.pyx
[3/5] Cythonizing /usr/lib64/python3.6/site-packages/urh/dev/native/lib/plutosdr.pyx
[4/5] Cythonizing /usr/lib64/python3.6/site-packages/urh/dev/native/lib/rtlsdr.pyx
[5/5] Cythonizing /usr/lib64/python3.6/site-packages/urh/dev/native/lib/usrp.pyx
running clean
removing 'build/lib.linux-x86_64-3.6' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.6' does not exist -- can't clean it
removing 'build'
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Init Process
[INFO::AbstractBaseThread.py::initialize_process] Starting Gnuradio
[DEBUG::AbstractBaseThread.py::initialize_process] /usr/bin/python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/hackrf_recv.py --samplerate 2000000.0 --freq 314500000.0 --gain 0 --bandwidth 2000000.0 --port 1337 --if-gain 16 --baseband-gain 16
[INFO::AbstractBaseThread.py::initialize_process] Started Gnuradio
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Process Intialized
[INFO::AbstractBaseThread.py::init_recv_socket] Initalizing receive socket
[INFO::AbstractBaseThread.py::init_recv_socket] Initalized receive socket
[INFO::AbstractBaseThread.py::init_recv_socket] Trying to get a connection to gnuradio...
[INFO::AbstractBaseThread.py::init_recv_socket] Got connection
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Socket initialized
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Enter main loop
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/urh/dev/gr/SpectrumThread.py", line 32, in run
    rcvd += recv(32768)  # Receive Buffer = 32768 Byte
  File "zmq/backend/cython/socket.pyx", line 693, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:7692)
  File "zmq/backend/cython/socket.pyx", line 727, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:7469)
  File "zmq/backend/cython/socket.pyx", line 150, in zmq.backend.cython.socket._recv_copy (zmq/backend/cython/socket.c:2446)
  File "zmq/backend/cython/socket.pyx", line 145, in zmq.backend.cython.socket._recv_copy (zmq/backend/cython/socket.c:2353)
  File "zmq/backend/cython/checkrc.pxd", line 19, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:9908)
zmq.error.Again: Resource temporarily unavailable
Aborted
pentoo-usb ~ #

The problematic seems code where a system-installed urh with python3 by default is trying to run a python2

blshkv commented 5 years ago

Steps To Reproduce bash$ urh

Edit->Options->Device

Disable "Native Backend" and use "Gnu Radio" only

File > Spectrum Analyzer > Start


[DEBUG::SpectrumThread.py::run] Spectrum Thread: Init Process
[INFO::AbstractBaseThread.py::initialize_process] Starting Gnuradio
[DEBUG::AbstractBaseThread.py::initialize_process] /usr/bin/python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/usrp_recv.py --samplerate 1000000.0 --freq 433920000.0 --gain 50 --bandwidth 1000000.0 --port 1337
[INFO::AbstractBaseThread.py::initialize_process] Started Gnuradio
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Process Intialized
[INFO::AbstractBaseThread.py::init_recv_socket] Initalizing receive socket
[INFO::AbstractBaseThread.py::init_recv_socket] Initalized receive socket
[INFO::AbstractBaseThread.py::init_recv_socket] Trying to get a connection to gnuradio...
[INFO::AbstractBaseThread.py::init_recv_socket] Got connection
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Socket initialized
[DEBUG::SpectrumThread.py::run] Spectrum Thread: Enter main loop
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/urh/dev/gr/SpectrumThread.py", line 32, in run
    rcvd += recv(32768)  # Receive Buffer = 32768 Byte
  File "zmq/backend/cython/socket.pyx", line 693, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:7692)
  File "zmq/backend/cython/socket.pyx", line 727, in zmq.backend.cython.socket.Socket.recv (zmq/backend/cython/socket.c:7469)
  File "zmq/backend/cython/socket.pyx", line 150, in zmq.backend.cython.socket._recv_copy (zmq/backend/cython/socket.c:2446)
  File "zmq/backend/cython/socket.pyx", line 145, in zmq.backend.cython.socket._recv_copy (zmq/backend/cython/socket.c:2353)
  File "zmq/backend/cython/checkrc.pxd", line 19, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:9908)
zmq.error.Again: Resource temporarily unavailable
QSocketNotifier: Invalid socket 6 and type 'Read', disabling...
KCrash: Application 'python3.6m' crashing...
KCrash: Attempting to start /usr/lib64/libexec/drkonqi from kdeinit
sock_file=/var/run/user/1000/kdeinit5__0
QSocketNotifier: Invalid socket 12 and type 'Read', disabling...
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.6/site-packages/urh/dev/gr/AbstractBaseThread.py", line 270, in enqueue_output
    for line in iter(out.readline, b''):
OSError: [Errno 9] Bad file descriptor

Bad file descriptor (src/epoll.cpp:88)
Unable to start Dr. Konqi

ZeroChaos- commented 5 years ago

The specific "silly thing" I was referencing was this:

[DEBUG::AbstractBaseThread.py::initialize_process] /usr/bin/python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/usrp_recv.py --samplerate 1000000.0 --freq 433920000.0 --gain 50 --bandwidth 1000000.0 --port 1337

calling something installed in the python 3 path but using python2. Perhaps the issue is on our side and we should be installing urh as python2?

blshkv commented 5 years ago

No, urh does not support python 2 officially. See setup.py : print("You need at least Python 3.4

jopohl commented 5 years ago

Now I see! Here is what is going on:

URH requires Python 3.4+ to work, true!
GNU Radio, at time of developing the GNU Radio backend, only supported Python 2.7.
Some distributions (e.g. Arch Linux) still install GNU Radio as a Python 2.7 lib, thats why the problem may be Gentoo/Pentoo specific (I did not check other distributions, apart from Arch). A solution might be to install GNU Radio as a Python 2 lib on your side? I am, however, not sure if this breaks other applications.

The most easy way to deal with this would be to enter a python 3 interpreter in the options:

bildschirmfoto_2018-12-18_08-30-28

blshkv commented 5 years ago

@jopohl we follow each package requirements. In particular, urh was installed as python3 module only and gnuradio as python2 (which seems irrelevant here).

I think you need to change that menu to:

Python3 interpreter = /usr/bin/python3

as a default option

jopohl commented 5 years ago

@blshkv This would break other distributions such as Arch and Ubuntu which I just checked. Python 3 support will be added in GR 3.8 according to here, while the latest official release for GR is 3.7.13.4. So I won't change the default option here, because it is furthermore documented, that the python 2 support for GR will be kept in later releases.

Just to be sure how you installed GR which of these lines does work:

python2 -c "import gnuradio"

or

python3 -c "import gnuradio"

blshkv commented 5 years ago

bash~ $ python2 -c "import gnuradio"
bash~ $ python3 -c "import gnuradio"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'gnuradio'

/usr/lib64/python3.6/site-packages/urh
/usr/lib64/python2.7/site-packages/gnuradio

jopohl commented 5 years ago

In that case, you also installed GNU Radio as a python 2 module, so how is it supposed to work overall like requested in this issue? Especially this "silly" line

[DEBUG::AbstractBaseThread.py::initialize_process] /usr/bin/python2 /usr/lib64/python3.6/site-packages/urh/dev/gr/scripts/usrp_recv.py --samplerate 1000000.0 --freq 433920000.0 --gain 50 --bandwidth 1000000.0 --port 1337

can't work any other way, because GNU Radio is installed as a Python 2 lib.

blshkv commented 5 years ago

that exactly the problem. So we either have to wait for GR 3.8 or make URH support python2 (together with python3 of course). What should it be?

jopohl commented 5 years ago

How is this a problem? You call a python 2 script with a python 2 interpreter. The GR backend works on many distributions including Arch and Ubuntu in the way it is now. I don't know what is the exact problem on Gentoo but it has nothing to do with Python 2/3.

Furthermore, what is the exact reason to use the GR backend in the user's case? Every device (except for FUN Cube dongle) has native support, so there is no need to use the GR backend, because the native one is faster anyway. This is also the reason why the native backend is marked as recommended in the GUI.

ZeroChaos- commented 5 years ago

it sounds like my assessment of the issue was incorrect, but there is an issue here for sure.

see details in https://github.com/jopohl/urh/issues/587#issuecomment-448038640

jopohl commented 5 years ago

Oh yeah sure, I was a bit mislead by the python 2/3 topic. Fix is on the way, PR is open, and will be merged in master soon.

jopohl commented 5 years ago

Just merged the fix in master. Can you test it on your end? If everything works I will create a new release.

jopohl commented 5 years ago

New release with included fix is out. I am closing here.

ZeroChaos- commented 5 years ago

my user confirmed it's working, thanks!

jopohl / urh