search

coiled / dask-bigquery

BSD 3-Clause "New" or "Revised" License
43 stars 13 forks source link

macOS tests seem to fail on `test_read_required_partition_filter` reliably #67

Closed j-bennet closed 1 year ago

j-bennet commented 1 year ago

Something is going on with test_read_required_partition_filter on mac, attempt to skip.

j-bennet commented 1 year ago

@bnaul I don't know what's so special with test_read_required_partition_filter, but it seems to segfault in CI on macos. Do you have any ides?

bnaul commented 1 year ago

huh, not sure...I'm on macos and it works for me locally

(.venv)  ➜  dask-bigquery git:(main) ✗ pytest -k test_read_required_partition_filter
========================================== test session starts ===========================================
platform darwin -- Python 3.10.11, pytest-7.3.1, pluggy-1.0.0
rootdir: /Users/brett/dask-bigquery
collected 17 items / 16 deselected / 1 selected

dask_bigquery/tests/test_core.py .                                                                 [100%]

=================================== 1 passed, 16 deselected in 15.34s ====================================
j-bennet commented 1 year ago

huh, not sure...I'm on macos and it works for me locally

Same here, but mine is M2, that may make a difference?

j-bennet commented 1 year ago

It just keeps failing with the same error every time.

https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66

dask_bigquery/tests/test_core.py::test_max_streams PASSED                [ 88%]
dask_bigquery/tests/test_core.py::test_arrow_options PASSED              [ 94%]
Fatal Python error: Segmentation fault
dask_bigquery/tests/test_core.py::test_read_required_partition_filter 

Thread 0x000070000da7d000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x000070000c5df000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x000070000b5dc000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x000070000a556000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007000093ca000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/selectors.py", line 562 in select
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/asyncio/base_events.py", line 1869 in _run_once
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/asyncio/base_events.py", line 601 in run_forever
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Thread 0x00007000083c7000 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/concurrent/futures/thread.py", line 81 in _worker
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 917 in run
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 980 in _bootstrap_inner
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/threading.py", line 937 in _bootstrap

Current thread 0x0000000114372600 (most recent call first):
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/grpc/_channel.py", line 1017 in _blocking
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/grpc/_channel.py", line 1028 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 72 in error_remapped_callable
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/api_core/timeout.py", line 120 in func_with_timeout
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/api_core/retry.py", line 191 in retry_target
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/api_core/retry.py", line [34](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:35)9 in retry_wrapped_func
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 113 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/google/cloud/bigquery_storage_v1/services/big_query_read/client.py", line 6[36](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:37) in create_read_session
  File "/Users/runner/work/dask-bigquery/dask-bigquery/dask_bigquery/core.py", line 189 in read_gbq
  File "/Users/runner/work/dask-bigquery/dask-bigquery/dask_bigquery/tests/test_core.py", line 3[39](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:40) in test_read_required_partition_filter
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/python.py", line 1799 in runtest
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 3[41](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:42) in from_call
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/main.py", line 3[48](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:49) in pytest_runtestloop
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/main.py", line 323 in _main
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/config/__init__.py", line 166 in main
  File "/Users/runner/miniconda3/envs/test-environment/lib/python3.9/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/Users/runner/miniconda3/envs/test-environment/bin/pytest", line 10 in <module>
/Users/runner/work/_temp/1e800241-22e9-4e6e-a0d3-5cba1[59](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:60)bc[61](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:62)5.sh: line 1:  91[64](https://github.com/coiled/dask-bigquery/actions/runs/4996647617/jobs/8950099702?pr=66#step:6:65) Segmentation fault: 11  pytest -v dask_bigquery
bnaul commented 1 year ago

skipping on mac seems fine to me