soft-matter / trackpy

Python particle tracking toolkit
http://soft-matter.github.io/trackpy
Other
445 stars 131 forks source link

Fix segfault after pytest on Py3.7, pip environment #655

Closed caspervdw closed 3 years ago

caspervdw commented 3 years ago

See: https://github.com/soft-matter/trackpy/runs/2236528914 and discussion in #654

The command pytest trackpy ends with:

=========== 929 passed, 17 skipped, 7 warnings in 353.98s (0:05:53) ============
Fatal Python error: Segmentation fault

Current thread 0x00007f14cdbda740 (most recent call first):
/home/runner/work/_temp/1ef82e28-b658-4675-8755-17c6857f00fb.sh: line 1:  2060 Segmentation fault      (core dumped) pytest trackpy
Error: Process completed with exit code 139.

So the segfault happens probably when the interpreter shuts down.

caspervdw commented 3 years ago

I was able to reproduce locally with the following dockerfile:

FROM python:3.7

RUN apt-get update && apt-get install -y libhdf5-dev --no-install-recommends

RUN pip install --disable-pip-version-check --upgrade pip && pip install --upgrade wheel 
RUN pip install --no-build-isolation numpy==1.16.*
RUN pip install --no-build-isolation pytest scipy==1.3.* matplotlib==2.2.* pillow==5.3.* pandas==0.23.* scikit-image==0.14.* tables==3.5.* scikit-learn==0.20.* pyyaml==3.13 numba==0.39.* llvmlite==0.24.*

COPY . /code

WORKDIR /code

The problems are caused by numba. I started to gradually bump numpy and they dissappeared with numba==0.45. I think this has to do with https://github.com/numba/numba/issues/4323 ; so an incompatibilty between numba <0.45 and python 3.7.4 (we test with 3.7.10, but I guess the incompatibility persists accross these micro versions).

Solution is: test with numba 0.45 in our py37 env.

rbnvrw commented 3 years ago

Sounds good. Cool that you found it so fast!