search

JnyJny / busylight

Control USB connected presence lights from multiple vendors via the command-line or web API.
Apache License 2.0
227 stars 25 forks source link

Receiving Segmentation fault when running 'busylight on' #228

Closed thereallogan5 closed 1 year ago

thereallogan5 commented 1 year ago 
Fatal Python error: Segmentation fault

Current thread 0x00007fb45f239000 (most recent call first):
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/hidlight.py", line 37 in available_lights
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/light.py", line 155 in all_lights
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/light.py", line 152 in all_lights
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 103 in lights
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 125 in selected_lights
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 196 in on
  File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/__main__.py", line 151 in turn_lights_on
  File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/main.py", line 683 in wrapper
  File "/usr/lib/python3/dist-packages/click/core.py", line 754 in invoke
  File "/usr/lib/python3/dist-packages/click/core.py", line 1395 in invoke
  File "/usr/lib/python3/dist-packages/click/core.py", line 1659 in invoke
  File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/core.py", line 216 in _main
  File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/core.py", line 778 in main
  File "/usr/lib/python3/dist-packages/click/core.py", line 1128 in __call__
  File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/main.py", line 311 in __call__
  File "/home/mrrobot/.local/bin/busylight", line 9 in <module>

Extension modules: hid (total: 1)
Segmentation fault (core dumped)
JnyJny commented 1 year ago

Could you give me a little more information about the host the SEGFAULT is occurring on? Some idea of which light, OS and CPU architecture would be helpful.

I'm curious what the result of this small python program is on your host in the environment that busylight is installed in:

import hid

for device in hid.enumerate():
    print(device)

This is the code that is SEGFAULT'ing at line 37 in busylight and is part of the hidapi package, which is a dependency.

There is a bug which might be applicable to your situation, but it's a guess without more context. ~Also, to be clear, busylight uses asyncio for its concurrency model which is not multithreaded (pretty sure anyway).~ Asyncio does use threads under the covers but as long as coroutines are called from the same event loop that initializes HID API it should avoid tickling the linked bug.

thereallogan5 commented 1 year ago

Hi, sure, Thanks,

Linux Mint 21.1 Vera Linux Vampiro 5.15.0-60-generic #66-Ubuntu SMP Fri Jan 20 14:29:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Was previously working.

python3 Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import hid for device in hid.enumerate(): ... print(device) ... {'path': b'3-3.5:1.0', 'vendor_id': 16700, 'product_id': 45166, 'serial_number': '', 'release_number': 257, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-3.3.5:1.0', 'vendor_id': 16700, 'product_id': 45167, 'serial_number': '', 'release_number': 257, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-3.3.2.2.1:1.0', 'vendor_id': 1133, 'product_id': 50475, 'serial_number': '', 'release_number': 9233, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-3.3.2.2.1:1.1', 'vendor_id': 1133, 'product_id': 50475, 'serial_number': '', 'release_number': 9233, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 1} {'path': b'3-3.3.2.2.1:1.2', 'vendor_id': 1133, 'product_id': 50475, 'serial_number': '', 'release_number': 9233, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 2} {'path': b'3-3.3.1:1.0', 'vendor_id': 11277, 'product_id': 12, 'serial_number': '', 'release_number': 256, 'manufacturer_string': '', 'product_string': 'Blynclight Plus', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-3.2:1.0', 'vendor_id': 1133, 'product_id': 49174, 'serial_number': '', 'release_number': 832, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-3.1.4:1.3', 'vendor_id': 1133, 'product_id': 2767, 'serial_number': '', 'release_number': 3589, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 3} {'path': b'3-3.1.3:1.3', 'vendor_id': 3468, 'product_id': 20, 'serial_number': '', 'release_number': 256, 'manufacturer_string': '', 'product_string': '', 'usage_page': 0, 'usage': 0, 'interface_number': 3} {'path': b'3-2:1.0', 'vendor_id': 4176, 'product_id': 1031, 'serial_number': '', 'release_number': 1316, 'manufacturer_string': 'Yubico', 'product_string': 'YubiKey OTP+FIDO+CCID', 'usage_page': 0, 'usage': 0, 'interface_number': 0} {'path': b'3-2:1.1', 'vendor_id': 4176, 'product_id': 1031, 'serial_number': '', 'release_number': 1316, 'manufacturer_string': 'Yubico', 'product_string': 'YubiKey OTP+FIDO+CCID', 'usage_page': 0, 'usage': 0, 'interface_number': 1}

JnyJny commented 1 year ago

Weird, I totally expected that to SEGFAULT and would definitively point towards hidapi as the culprit here.

So when you said, "previously working" could you fill in some context? Did hardware/software change or did it just stop working all of a sudden? Could you elaborate more about how you installed busylight and also the version that is crashing?

Also could you run the crashing command with the -D flag and post the output?

$ busylight -D on

If a core file is generated, that could potentially give us more information about what's going wrong as well. If you have gdb installed on your machine, you could use that to inspect the core file. A stack trace is what we're after, something like:

$ gdb `path/to/python3` path/to/corefile
(gdb) bt

To be honest, it's been a while since I've had to do this sort of debugging on Linux so all of that might not be strictly correct but the vague shape of how to get a back trace from the core file.

I know that's a lot to ask for but I haven't seen this kind of failure before and I haven't been able to replicate it yet on the only Linux box I have handy (RaspPi).

JnyJny commented 1 year ago

So I'm considering that this might be an asyncio problem, does the following short python snippet crash?

from busylight.lights import Blynclight

light = Blynclight.first_light()
light.on((0,255,0)) # turn on green
thereallogan5 commented 1 year ago

Hi JnyJny,

Was previously working for several days fine but not sure what caused it to stop working, whether it was an update, (there's been so many lately).

Here's the output:

busylight -D on 2023-02-21 18:02:28.336 | INFO | busylight.main:precommand_callback:132 - version 0.25.3 2023-02-21 18:02:28.337 | INFO | busylight.main:precommand_callback:133 - timeout=None 2023-02-21 18:02:28.337 | INFO | busylight.main:precommand_callback:134 - dim=1.0 2023-02-21 18:02:28.337 | INFO | busylight.main:precommand_callback:135 - lights=[0] 2023-02-21 18:02:28.337 | INFO | busylight.main:precommand_callback:136 - cmd='on' 2023-02-21 18:02:28.337 | INFO | busylight.main:turn_lights_on:149 - activating lights Fatal Python error: Segmentation fault

Current thread 0x00007f8d1805d000 (most recent call first): File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/hidlight.py", line 37 in available_lights File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/light.py", line 155 in all_lights File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/lights/light.py", line 152 in all_lights File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 103 in lights File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 125 in selected_lights File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/manager.py", line 196 in on File "/home/mrrobot/.local/lib/python3.10/site-packages/busylight/main.py", line 151 in turn_lights_on File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/main.py", line 683 in wrapper File "/usr/lib/python3/dist-packages/click/core.py", line 754 in invoke File "/usr/lib/python3/dist-packages/click/core.py", line 1395 in invoke File "/usr/lib/python3/dist-packages/click/core.py", line 1659 in invoke File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/core.py", line 216 in _main File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/core.py", line 778 in main File "/usr/lib/python3/dist-packages/click/core.py", line 1128 in call File "/home/mrrobot/.local/lib/python3.10/site-packages/typer/main.py", line 311 in call File "/home/mrrobot/.local/bin/busylight", line 9 in

Extension modules: hid (total: 1) Segmentation fault (core dumped)

thereallogan5 commented 1 year ago

...and the output from the snippet:

python3
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from busylight.lights import Blynclight
>>> 
>>> light = Blynclight.first_light()
Segmentation fault (core dumped)
thereallogan5 commented 1 year ago

(can't seem to find a dump file when running busylight -D on from $HOME, searched in .local/bin but nothing in there either.)

JnyJny commented 1 year ago

Well, that is interesting. There is no use of asyncio in that batch of code, so it's less likely that the problem is asyncio tickling an edge case in hidapi. Now I'm starting to think about how busylight is installed and if there is a mismatched dependency somewhere. I see that the busylight command-line script is installed in $HOME/.local/bin/busylight while click and typer are being imported from /usr/lib/python3/dist-packages, and the busylight package is imported from $HOME/.local/lib/python3.10/site-packages. That might be okay, but it might not be, it's hard to tell without rummaging around in the installation.

As an experiment, could you try this?

$ cd /tmp
$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install busylight
$ which busylight # should say something like /tmp/venv/bin/busylight
$ busylight -D on
$ deactivate # to turn off this virtual environment
$ rm /tmp/venv # to get rid of the virtual environment when you are done

I'm hoping that a fresh install of all the requirements into a virtual environment will allow busylight to execute without a SEGFAULT.

thereallogan5 commented 1 year ago

hmmh, can't seem to get the installation to run in the venv:

mrrobot@Vampiro:/tmp$ python3 -m venv venv
mrrobot@Vampiro:/tmp$ source venv/bin/activate
(venv) mrrobot@Vampiro:/tmp$ python3 -m pip install busylight
ERROR: Could not find a version that satisfies the requirement busylight (from versions: none)
ERROR: No matching distribution found for busylight
JnyJny commented 1 year ago

Interesting.. what version of python did you use to create the virtual environment?

$ python3 --version
$ which python3
thereallogan5 commented 1 year ago 
python3 --version
Python 3.10.6
mrrobot@Vampiro:/tmp$ which python3
/usr/bin/python3
JnyJny commented 1 year ago

Well this is embarrassing, let's try:

$ cd /tmp
$ python3 -m venv venv
$ source venv/bin/activate
$ python3 -m pip install busylight-for-humans
$ which busylight
$ busylight on

I rarely install from PyPI and forgot that the proper name for busylight is busylight-for-humans.

thereallogan5 commented 1 year ago

Well that went further till the last command:

mrrobot@Vampiro:/tmp$ python3 --version
Python 3.10.6
mrrobot@Vampiro:/tmp$ which python3
/usr/bin/python3
mrrobot@Vampiro:/tmp$ python3 -m venv venv
mrrobot@Vampiro:/tmp$ source venv/bin/activate
(venv) mrrobot@Vampiro:/tmp$ python3 -m pip install busylight-for-humans
Collecting busylight-for-humans
  Using cached busylight_for_humans-0.25.3-py3-none-any.whl (54 kB)
Collecting pyserial<4.0,>=3.5
  Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
Collecting webcolors<2.0.0,>=1.11.1
  Using cached webcolors-1.12-py3-none-any.whl (9.9 kB)
Collecting loguru<0.7.0,>=0.6.0
  Using cached loguru-0.6.0-py3-none-any.whl (58 kB)
Collecting bitvector-for-humans<0.15.0,>=0.14.0
  Using cached bitvector_for_humans-0.14.1-py3-none-any.whl (10 kB)
Collecting typer<0.8.0,>=0.7.0
  Using cached typer-0.7.0-py3-none-any.whl (38 kB)
Collecting hidapi<0.14.0,>=0.13.1
  Using cached hidapi-0.13.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (906 kB)
Requirement already satisfied: setuptools>=19.0 in ./venv/lib/python3.10/site-packages (from hidapi<0.14.0,>=0.13.1->busylight-for-humans) (59.6.0)
Collecting click<9.0.0,>=7.1.1
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 929.2 kB/s eta 0:00:00
Installing collected packages: pyserial, webcolors, loguru, hidapi, click, bitvector-for-humans, typer, busylight-for-humans
Successfully installed bitvector-for-humans-0.14.1 busylight-for-humans-0.25.3 click-8.1.3 hidapi-0.13.1 loguru-0.6.0 pyserial-3.5 typer-0.7.0 webcolors-1.12
(venv) mrrobot@Vampiro:/tmp$ which busylight
/tmp/venv/bin/busylight
(venv) mrrobot@Vampiro:/tmp$ busylight on
Segmentation fault (core dumped)
thereallogan5 commented 1 year ago

Tried this way with -D

busylight -D on
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:precommand_callback:132 - version 0.25.3
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:precommand_callback:133 - timeout=None
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:precommand_callback:134 -     dim=1.0
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:precommand_callback:135 -  lights=[0]
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:precommand_callback:136 -     cmd='on'
2023-02-22 11:36:43.433 | INFO     | busylight.__main__:turn_lights_on:149 - activating lights
Segmentation fault (core dumped)
JnyJny commented 1 year ago

Thanks for the install output! I went through the versions of the installed dependencies and they all track with what I think should be installed.

Did you set up/modify the udev rules for your Linux installation? If so, can you share the contents of your rules?

thereallogan5 commented 1 year ago

NP!
I had run through the udev step earlier

# Generated by `busylight udev-rules` https://github.com/JnyJny/busylight
#
# Rules for Agile Innovative Family of Devices: 1
# 1 Agile Innovative BlinkStick
KERNEL=="hidraw*", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="41e5", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="41e5", MODE="0666"
# Rules for Embrava Family of Devices: 3
# 1 Embrava Blynclight
KERNEL=="hidraw*", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="0001", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="0001", MODE="0666"
# 2 Embrava Blynclight
KERNEL=="hidraw*", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="000c", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="000c", MODE="0666"
# 3 Embrava Blynclight
KERNEL=="hidraw*", ATTRS{idVendor}=="0e53", ATTRS{idProduct}=="2516", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0e53", ATTRS{idProduct}=="2516", MODE="0666"
# Rules for Embrava Family of Devices: 2
# 1 Embrava Blynclight Mini
KERNEL=="hidraw*", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="000a", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="000a", MODE="0666"
# 2 Embrava Blynclight Mini
KERNEL=="hidraw*", ATTRS{idVendor}=="0e53", ATTRS{idProduct}=="2517", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0e53", ATTRS{idProduct}=="2517", MODE="0666"
# Rules for Embrava Family of Devices: 1
# 1 Embrava Blynclight Plus
KERNEL=="hidraw*", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="0010", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2c0d", ATTRS{idProduct}=="0010", MODE="0666"
# Rules for Plantronics Family of Devices: 1
# 1 Plantronics Status Indicator
KERNEL=="hidraw*", ATTRS{idVendor}=="047f", ATTRS{idProduct}=="d005", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="047f", ATTRS{idProduct}=="d005", MODE="0666"
# Rules for Kuando Family of Devices: 3
# 1 Kuando Busylight Alpha
KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f848", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f848", MODE="0666"
# 2 Kuando Busylight Alpha
KERNEL=="hidraw*", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bca", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bca", MODE="0666"
# 3 Kuando Busylight Alpha
KERNEL=="hidraw*", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bce", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bce", MODE="0666"
# Rules for Kuando Family of Devices: 2
# 1 Kuando Busylight Omega
KERNEL=="hidraw*", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bcd", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bcd", MODE="0666"
# 2 Kuando Busylight Omega
KERNEL=="hidraw*", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bcf", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="27bb", ATTRS{idProduct}=="3bcf", MODE="0666"
# Rules for Luxafor Family of Devices: 1
# 1 Luxafor Flag
KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
# Rules for Luxafor Family of Devices: 1
# 1 Luxafor BT
KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
# Rules for Luxafor Family of Devices: 1
# 1 Luxafor Mute
KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
# Rules for Luxafor Family of Devices: 1
# 1 Luxafor Orb
KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f372", MODE="0666"
# Rules for MuteMe Family of Devices: 2
# 1 MuteMe MuteMe Original
KERNEL=="hidraw*", ATTRS{idVendor}=="16c0", ATTRS{idProduct}=="27db", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="16c0", ATTRS{idProduct}=="27db", MODE="0666"
# 2 MuteMe MuteMe Original
KERNEL=="hidraw*", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="42da", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="42da", MODE="0666"
# Rules for MuteMe Family of Devices: 1
# 1 MuteMe MuteMe Mini
KERNEL=="hidraw*", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="42db", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="20a0", ATTRS{idProduct}=="42db", MODE="0666"
# Rules for ThingM Family of Devices: 1
# 1 ThingM Blink(1)
KERNEL=="hidraw*", ATTRS{idVendor}=="27b8", ATTRS{idProduct}=="01ed", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="27b8", ATTRS{idProduct}=="01ed", MODE="0666"
JnyJny commented 1 year ago

I don't see anything wrong with the udev rules, so it's probably time to start thinking about finding the core file and trying to get a stack trace to see a more low level description of how the code crashed. Before we start looking for and at the core file, can you save the output of the dmesg command and the contents of /var/log/messages and attach them here? Those two things might give us more information about what the failure is and maybe what's causing it.

thereallogan5 commented 1 year ago

dmesg.txt

thereallogan5 commented 1 year ago

I don't seem to have a 'messages' in /var/log

JnyJny commented 1 year ago

/var/log/messages is one location where syslog messages are stored, no worries.

Looking at grep usb dmesg.txt it appears that you have a Dell notebook dock acting as a USB hub with many devices attached. There are several errors complaining about being unable to enumerate USB devices (where the code is crashing) and errors associated with the Blue Yeti Nano attached to the dock.

My thoughts are leading me towards a hardware problem (bad hub, bad device or bad cable). I'd unplug everything and plug the Blynclight in directly to a USB port (if possible) and see if that causes a crash.

thereallogan5 commented 1 year ago

Hi JnyJny,

Thanks for the feedback!, I'll try that and let you know.

thereallogan5 commented 1 year ago

Hi JnyJny,

Many thanks for the excellent troubleshooting and busylight! Turned out to be the docking station/hub issue after receiving several updates. Plugging directly in has resolved.

Best, -L

JnyJny commented 1 year ago

That is great news! I was running out of ideas TBH :) Sorry you are having trouble with your docking station and I hope you get it sorted out.