vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.3k stars 591 forks source link

[BUG-REPORT] Hitting `pybind11::handle::inc_ref()` when using NumPy 1.26 with vaex 4.18.0 #2439

Open setu4993 opened 1 month ago

setu4993 commented 1 month ago

Description

All our unit tests and validations which worked fine on v4.17.1 started failing upon updating to v4.18.0. Curiously, this is occurring only on GitHub Actions so far (ubuntu-latest images) and not reproducible on M-series MacBooks. I suspect this is limited to some instruction sets.

I don't have a great sample right now (but can get one tomorrow) to reproduce this, but the CI run here might be helpful to compare the diff. This is occurring across repos but this public one was the easiest to link to.

Software information

iisakkirotko commented 1 month ago

I think we're running into the same issue in this run at widgetti/solara. We also see an error on windows: Windows fatal exception: access violation resulting in a seg fault.

The full log for Windows ```bash Thread 0x00000694 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\hash.py", line 171 in add File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\cpu.py", line 344 in process File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.py", line 564 in process_tasks File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.py", line 500 in process_part File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\multithreading.py", line 80 in wrapped File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\thread.py", line 58 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\thread.py", line 83 in _worker File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 917 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Current thread 0x00000b14 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\hash.py", line 171 in add File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\cpu.py", line 344 in process File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.py", line 564 in process_tasks File "Windows fatal exception: C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.pyaccess violation" , line 500 in process_part File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\multithreading.py", line 80 in wrapped File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\thread.py", line 58 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\thread.py", line 83 in _worker File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 917 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001810 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 316 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 581 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 1304 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001468 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000004cc (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line [38](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:39)0 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000a44 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000019ec (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x0000130c (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000798 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000b18 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001ff0 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001074 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000958 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001a84 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000f34 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000018e4 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001b6c (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000013e4 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000e08 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000c94 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000009b0 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001514 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001454 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001220 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000614 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001a6c (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000c18 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000c1c (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000015f8 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000006ec (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x000002d0 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001058 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001510 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001a40 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000334 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001758 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\queue.py", line 171 in get File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 379 in dispatch_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 213 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00000d24 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 326 in read_directory_changes File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\winapi.py", line 380 in read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 67 in _read_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\read_directory_changes.py", line 70 in queue_events File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\watchdog\observers\api.py", line 158 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 306 in _WidgetContextAwareThread__bootstrap File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\patch.py", line 284 in WidgetContextAwareThread__bootstrap Thread 0x00001de4 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 581 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\IPython\core\history.py", line 894 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\IPython\core\history.py", line 60 in only_when_enabled File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\decorator.py", line 232 in fun File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap Thread 0x00000468 (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\asyncio\windows_events.py", line 788 in _poll File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\asyncio\windows_events.py", line 4[39](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:40) in select File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\asyncio\base_events.py", line 1869 in _run_once File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\asyncio\base_events.py", line 601 in run_forever File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\asyncio\windows_events.py", line 321 in run_forever File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\server\kernel_context.py", line 290 in _run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 917 in run File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 980 in _bootstrap_inner File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 937 in _bootstrap Thread 0x0000095c (most recent call first): File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\threading.py", line 312 in wait File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\_base.py", line 4[41](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:42) in result File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\concurrent\futures\_base.py", line 609 in result_iterator File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\itertools.py", line 5 in buffer File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\multithreading.py", line 104 in map File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.py", line 432 in execute_generator File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\execution.py", line 308 in execute File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\dataframe.py", line [42](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:43)1 in execute File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\dataframe.py", line 1780 in _delay File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\vaex\dataframe.py", line 7194 in groupby File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\lab\utils\dataframe.py", line 63 in df_value_count File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\solara\components\cross_filter.py", line 82 in CrossFilterSelect File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 1707 in _render File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2180 in _visit_children_values File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2183 in File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2183 in _visit_children_values File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2188 in File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2188 in _visit_children_values File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2175 in _visit_children File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 1625 in _render File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 1742 in _render File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 1742 in _render File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 1[43](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:44)6 in render File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\reacton\core.py", line 2210 in render File "D:\a\solara\solara\tests\unit\cross_filter_component_test.py", line 36 in test_cross_filter_select File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\python.py", line 159 in pytest_pyfunc_call File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 103 in _multicall File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 513 in __call__ File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\python.py", line 1627 in runtest File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 174 in pytest_runtest_call File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 103 in _multicall File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line [51](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:52)3 in __call__ File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 242 in File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 341 in from_call File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 241 in call_and_report File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 132 in runtestprotocol File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 113 in pytest_runtest_protocol File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 103 in _multicall File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 513 in __call__ File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 362 in pytest_runtestloop File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 103 in _multicall File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 513 in __call__ File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 337 in _main File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 283 in wrap_session File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 330 in pytest_cmdline_main File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 103 in _multicall File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 513 in __call__ File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\config\__init__.py", line 175 in main File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\config\__init__.py", line 201 in console_main File "C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts\pytest.exe\__main__.py", line 7 in File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\runpy.py", line 87 in _run_code File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\runpy.py", line 197 in _run_module_as_main D:\a\_temp\691c007b-3684-4dbd-aff5-fd213c6997ea.sh: line 3: 9[56](https://github.com/widgetti/solara/actions/runs/11138736498/job/30955789237#step:9:57) Segmentation fault pytest tests/unit --doctest-modules --timeout=60 ```
maartenbreddels commented 1 month ago

The relevant part of the logs:

self = <vaex.hash.HashMapUnique object at 0x7fe128022ee0>

    def flatten(self):
        if self.dtype == object:
            return self # already flat
>       keys = self._internal.key_array()
E       RuntimeError: pybind11::handle::inc_ref() PyGILState_Check() failure.

/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/vaex/hash.py:78: RuntimeError
----------------------------- Captured stderr call -----------------------------
pybind11::handle::inc_ref() is being called while the GIL is either not held or invalid. Please see https://pybind11.readthedocs.io/en/stable/advanced/misc.html#common-sources-of-global-interpreter-lock-errors for debugging advice.
The failing pybind11::handle::inc_ref() call was triggered on a numpy.ndarray object.

I think this is a bug in vaex-core which only gets exposed now due to upgrading pybind11. We should write a test to expose this, and see if we can reproduce this in CI. I think we should yank the vaex-core release to avoid other people hitting this.

setu4993 commented 1 month ago

+1, yanking seems like a good idea if this is arising from vaex-core.

maartenbreddels commented 1 month ago

it's yanked

maartenbreddels commented 1 month ago

I could not reproduce this on a local ubuntu machine with Python 3.11, but i'll continue to see if I can reproduce it (help is welcome)

ddelange commented 1 month ago

I googled RuntimeError: pybind11::handle::inc_ref() PyGILState_Check() failure and blamed it onto the following release from 2022 (3rd bullet point):

https://github.com/pybind/pybind11/releases/tag/v2.10.2

looks like the end-user can disable these checks using environment variables although it does look like a valid point made in the PR description

maartenbreddels commented 1 month ago

Update: We do not build with -DNDEBUG when we use cibuildwheel, but in a regular build we do pass this to the compiler. This causes our release to be shipped with this check, while in our CI we do not have this check. This is the exact opposite of what we want.

I'm not sure why this is, and who sets this (layers of abstraction are not always great...)

maartenbreddels commented 1 month ago

See https://github.com/vaexio/vaex/pull/2440

ddelange commented 1 month ago

https://pypi.org/project/vaex-core/4.18.1/

setu4993 commented 1 month ago

Thank you!