search

talmolab / sleap

A deep learning framework for multi-animal pose tracking.
https://sleap.ai
Other
414 stars 97 forks source link

Deleting instance causes segmentation fault #969

Closed daskandalis closed 1 year ago

daskandalis commented 1 year ago

Bug description

Deleting instance causes segmentation fault on next action.

Expected behaviour

Added extra instance with Ctrl+I. Right clicked to hide all landmarks in the instance, which deleted the instance. Expected to continue digitising.

Actual behaviour

Slightly variable but may include:

Nothing appearing when adding instance to next frame.

Subsequent clicking causes segmentation fault and core dump.

Your personal set up

SLEAP: 1.2.8 TensorFlow: 2.8.3 Numpy: 1.21.5 Python: 3.7.13 OS: Linux-5.15.0-48-generic-x86_64-with-debian-bookworm-sid

Environment packages ``` # paste output of `pip freeze` or `conda list` here ``` # packages in environment at /home/usr/anaconda3/envs/sleap: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu absl-py 1.2.0 pypi_0 pypi alabaster 0.7.12 pypi_0 pypi anyio 3.6.1 pypi_0 pypi appdirs 1.4.4 pypi_0 pypi argon2-cffi 21.3.0 pypi_0 pypi argon2-cffi-bindings 21.2.0 pypi_0 pypi astunparse 1.6.3 pypi_0 pypi attrs 21.4.0 pypi_0 pypi babel 2.10.3 pypi_0 pypi backcall 0.2.0 pypi_0 pypi backports-zoneinfo 0.2.1 pypi_0 pypi beautifulsoup4 4.11.1 pypi_0 pypi black 21.6b0 pypi_0 pypi bleach 5.0.1 pypi_0 pypi bzip2 1.0.8 h7b6447c_0 ca-certificates 2022.07.19 h06a4308_0 cached-property 1.5.2 py_0 cachetools 5.2.0 pypi_0 pypi cattrs 1.1.1 pypi_0 pypi certifi 2021.10.8 pypi_0 pypi cffi 1.15.1 pypi_0 pypi cfgv 3.3.1 pypi_0 pypi charset-normalizer 2.1.1 pypi_0 pypi click 8.0.4 pypi_0 pypi colorama 0.4.5 pypi_0 pypi commonmark 0.9.1 pypi_0 pypi coverage 6.4.4 pypi_0 pypi cryptography 38.0.1 pypi_0 pypi cuda-nvcc 11.3.58 h2467b9f_0 nvidia cudatoolkit 11.3.1 h2bc3f7f_2 cudnn 8.2.1 cuda11.3_0 cycler 0.11.0 pypi_0 pypi dbus 1.13.18 hb2f20db_0 debugpy 1.6.3 pypi_0 pypi decorator 5.1.1 pypi_0 pypi defusedxml 0.7.1 pypi_0 pypi deprecated 1.2.13 pypi_0 pypi distlib 0.3.6 pypi_0 pypi docutils 0.17.1 pypi_0 pypi efficientnet 1.0.0 pypi_0 pypi entrypoints 0.4 pypi_0 pypi expat 2.4.4 h295c915_0 ffmpeg 4.2.2 h20bf706_0 filelock 3.8.0 pypi_0 pypi flatbuffers 2.0.7 pypi_0 pypi fontconfig 2.13.1 h6c09931_0 fonttools 4.37.3 pypi_0 pypi freetype 2.11.0 h70c0345_0 gast 0.5.3 pypi_0 pypi geos 3.8.0 he6710b0_0 gettext 0.21.0 hf68c758_0 giflib 5.2.1 h7b6447c_0 gitdb 4.0.9 pypi_0 pypi gitpython 3.1.27 pypi_0 pypi glib 2.68.2 h36276a3_0 gmp 6.2.1 h295c915_3 gnutls 3.6.15 he1e5248_0 google-auth 2.11.1 pypi_0 pypi google-auth-oauthlib 0.4.6 pypi_0 pypi google-pasta 0.2.0 pypi_0 pypi greenlet 1.1.3 pypi_0 pypi grpcio 1.49.1 pypi_0 pypi gst-plugins-base 1.14.5 h0935bb2_2 conda-forge gstreamer 1.18.4 h76c114f_2 conda-forge h5py 3.1.0 nompi_py37h1e651dc_100 conda-forge hdf5 1.10.6 h3ffc7dd_1 hdmf 3.4.5 pypi_0 pypi icu 68.1 h2531618_0 identify 2.5.5 pypi_0 pypi idna 3.4 pypi_0 pypi image-classifiers 1.0.0 pypi_0 pypi imageio 2.15.0 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi imgaug 0.4.0 pypi_0 pypi imgstore 0.2.9 pypi_0 pypi importlib-metadata 4.12.0 pypi_0 pypi importlib-resources 5.9.0 pypi_0 pypi iniconfig 1.1.1 pypi_0 pypi ipykernel 6.15.3 pypi_0 pypi ipython 7.34.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 7.7.2 pypi_0 pypi jaraco-classes 3.2.2 pypi_0 pypi jedi 0.17.2 pypi_0 pypi jeepney 0.8.0 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi joblib 1.2.0 pypi_0 pypi jpeg 9e h7f8727e_0 jsmin 3.0.1 pypi_0 pypi json5 0.9.10 pypi_0 pypi jsonpickle 1.2 pypi_0 pypi jsonschema 4.16.0 pypi_0 pypi jupyter-cache 0.4.3 pypi_0 pypi jupyter-client 7.3.5 pypi_0 pypi jupyter-core 4.11.1 pypi_0 pypi jupyter-server 1.13.5 pypi_0 pypi jupyter-server-mathjax 0.2.6 pypi_0 pypi jupyter-sphinx 0.3.2 pypi_0 pypi jupyterlab 3.3.4 pypi_0 pypi jupyterlab-pygments 0.2.2 pypi_0 pypi jupyterlab-server 2.15.1 pypi_0 pypi jupyterlab-widgets 1.1.1 pypi_0 pypi keras 2.8.0 pypi_0 pypi keras-applications 1.0.8 pypi_0 pypi keras-preprocessing 1.1.2 pypi_0 pypi keyring 23.9.3 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi krb5 1.19.2 hac12032_0 lame 3.100 h7b6447c_0 lcms2 2.12 h3be6417_0 ld_impl_linux-64 2.38 h1181459_1 lerc 3.0 h295c915_0 libblas 3.9.0 15_linux64_openblas conda-forge libcblas 3.9.0 15_linux64_openblas conda-forge libclang 14.0.6 pypi_0 pypi libdeflate 1.8 h7f8727e_5 libedit 3.1.20210910 h7f8727e_0 libevent 2.1.10 h9b69904_4 conda-forge libffi 3.3 he6710b0_2 libgcc-ng 11.2.0 h1234567_1 libgfortran-ng 11.2.0 h00389a5_1 libgfortran5 11.2.0 h1234567_1 libglib 2.68.2 h3e27bee_0 conda-forge libgomp 11.2.0 h1234567_1 libiconv 1.16 h7f8727e_2 libidn2 2.3.2 h7f8727e_0 liblapack 3.9.0 15_linux64_openblas conda-forge libllvm11 11.1.0 h9e868ea_5 libopenblas 0.3.20 h043d6bf_1 libopus 1.3.1 h7b6447c_0 libpng 1.6.37 hbc83047_0 libpq 12.9 h16c4e8d_3 libstdcxx-ng 11.2.0 h1234567_1 libtasn1 4.16.0 h27cfd23_0 libtiff 4.4.0 hecacb30_0 libunistring 0.9.10 h27cfd23_0 libuuid 1.0.3 h7f8727e_2 libvpx 1.7.0 h439df22_0 libwebp 1.2.2 h55f646e_0 libwebp-base 1.2.2 h7f8727e_0 libxcb 1.15 h7f8727e_0 libxkbcommon 1.0.3 he3ba5ed_0 conda-forge libxml2 2.9.12 h72842e0_0 conda-forge libxslt 1.1.33 h15afd5d_2 conda-forge linkify-it-py 2.0.0 pypi_0 pypi livereload 2.6.3 pypi_0 pypi lxml 4.9.1 pypi_0 pypi lz4-c 1.9.3 h295c915_1 markdown 3.4.1 pypi_0 pypi markdown-it-py 1.1.0 pypi_0 pypi markupsafe 2.1.1 pypi_0 pypi matplotlib 3.5.3 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mdit-py-plugins 0.2.8 pypi_0 pypi mistune 0.8.4 pypi_0 pypi more-itertools 8.14.0 pypi_0 pypi mypy-extensions 0.4.3 pypi_0 pypi mysql-common 8.0.25 ha770c72_2 conda-forge mysql-libs 8.0.25 hfa10184_2 conda-forge myst-nb 0.13.2 pypi_0 pypi myst-parser 0.15.2 pypi_0 pypi nbclassic 0.4.3 pypi_0 pypi nbclient 0.5.13 pypi_0 pypi nbconvert 6.5.3 pypi_0 pypi nbdime 3.1.1 pypi_0 pypi nbformat 5.1.3 pypi_0 pypi ncurses 6.3 h5eee18b_3 ndx-pose 0.1.1 pypi_0 pypi nest-asyncio 1.5.5 pypi_0 pypi nettle 3.7.3 hbbd107a_1 networkx 2.6.3 pypi_0 pypi nodeenv 1.7.0 pypi_0 pypi notebook 6.4.12 pypi_0 pypi notebook-shim 0.1.0 pypi_0 pypi nspr 4.33 h295c915_0 nss 3.74 h0370c37_0 numpy 1.21.5 py37h976b520_1 conda-forge oauthlib 3.2.1 pypi_0 pypi olefile 0.46 py37_0 opencv-python 4.5.5.64 pypi_0 pypi openh264 2.1.1 h4ff587b_0 openssl 1.1.1q h7f8727e_0 opt-einsum 3.3.0 pypi_0 pypi packaging 21.3 pyhd3eb1b0_0 pandas 1.3.5 py37he8f5f7f_0 conda-forge pandocfilters 1.5.0 pypi_0 pypi parso 0.7.1 pypi_0 pypi pathspec 0.10.1 pypi_0 pypi pcre 8.45 h295c915_0 pexpect 4.8.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 8.4.0 py37h5aabda8_0 pip 22.2.2 pyhd8ed1ab_0 conda-forge pkginfo 1.8.3 pypi_0 pypi pkgutil-resolve-name 1.3.10 pypi_0 pypi platformdirs 2.5.2 pypi_0 pypi pluggy 1.0.0 pypi_0 pypi pre-commit 2.20.0 pypi_0 pypi prometheus-client 0.14.1 pypi_0 pypi prompt-toolkit 3.0.31 pypi_0 pypi protobuf 3.19.5 pypi_0 pypi psutil 5.9.2 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi py 1.11.0 pypi_0 pypi pyasn1 0.4.8 pypi_0 pypi pyasn1-modules 0.2.8 pypi_0 pypi pycparser 2.21 pypi_0 pypi pydata-sphinx-theme 0.8.1 pypi_0 pypi pygithub 1.55 pypi_0 pypi pygments 2.13.0 pypi_0 pypi pyjwt 2.5.0 pypi_0 pypi pykalman 0.9.5 pypi_0 pypi pynacl 1.5.0 pypi_0 pypi pynwb 2.1.0 pypi_0 pypi pyparsing 3.0.9 py37h06a4308_0 pyrsistent 0.18.1 pypi_0 pypi pyside2 5.13.2 py37hfa98aef_7 conda-forge pytest 7.1.3 pypi_0 pypi pytest-cov 3.0.0 pypi_0 pypi pytest-qt 4.1.0 pypi_0 pypi pytest-xvfb 2.0.0 pypi_0 pypi python 3.7.13 h12debd9_0 python-dateutil 2.8.2 pyhd3eb1b0_0 python-rapidjson 1.8 pypi_0 pypi python_abi 3.7 2_cp37m conda-forge pytz 2022.1 py37h06a4308_0 pytz-deprecation-shim 0.1.0.post0 pypi_0 pypi pyvirtualdisplay 3.0 pypi_0 pypi pywavelets 1.3.0 pypi_0 pypi pyyaml 6.0 pypi_0 pypi pyzmq 24.0.1 pypi_0 pypi qimage2ndarray 1.9.0 pypi_0 pypi qt 5.12.9 h9d6b050_2 conda-forge qtpy 2.2.0 py37h06a4308_0 readline 8.1.2 h7f8727e_1 readme-renderer 37.2 pypi_0 pypi regex 2022.9.13 pypi_0 pypi requests 2.28.1 pypi_0 pypi requests-oauthlib 1.3.1 pypi_0 pypi requests-toolbelt 0.9.1 pypi_0 pypi rfc3986 2.0.0 pypi_0 pypi rich 10.16.1 pypi_0 pypi rsa 4.9 pypi_0 pypi ruamel-yaml 0.17.21 pypi_0 pypi ruamel-yaml-clib 0.2.6 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scikit-learn 1.0.2 pypi_0 pypi scikit-video 1.1.11 pypi_0 pypi scipy 1.7.3 py37hf2a6cf1_0 conda-forge seaborn 0.12.0 pypi_0 pypi secretstorage 3.3.3 pypi_0 pypi segmentation-models 1.0.1 pypi_0 pypi send2trash 1.8.0 pypi_0 pypi setuptools 58.0.4 py37h06a4308_0 shapely 1.7.1 py37h1728cc4_0 six 1.16.0 pyhd3eb1b0_1 sleap 1.2.8 dev_0 smmap 5.0.0 pypi_0 pypi sniffio 1.3.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi soupsieve 2.3.2.post1 pypi_0 pypi sphinx 4.5.0 pypi_0 pypi sphinx-autobuild 2021.3.14 pypi_0 pypi sphinx-book-theme 0.3.3 pypi_0 pypi sphinx-copybutton 0.5.0 pypi_0 pypi sphinx-togglebutton 0.3.2 pypi_0 pypi sphinxcontrib-applehelp 1.0.2 pypi_0 pypi sphinxcontrib-devhelp 1.0.2 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.0 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.3 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.5 pypi_0 pypi sqlalchemy 1.4.41 pypi_0 pypi sqlite 3.39.2 h5082296_0 tensorboard 2.8.0 pypi_0 pypi tensorboard-data-server 0.6.1 pypi_0 pypi tensorboard-plugin-wit 1.8.1 pypi_0 pypi tensorflow 2.8.3 pypi_0 pypi tensorflow-estimator 2.8.0 pypi_0 pypi tensorflow-io-gcs-filesystem 0.27.0 pypi_0 pypi termcolor 2.0.1 pypi_0 pypi terminado 0.15.0 pypi_0 pypi threadpoolctl 3.1.0 pypi_0 pypi tifffile 2021.11.2 pypi_0 pypi tinycss2 1.1.1 pypi_0 pypi tk 8.6.12 h1ccaba5_0 toml 0.10.2 pypi_0 pypi tomli 2.0.1 pypi_0 pypi tornado 6.2 pypi_0 pypi tqdm 4.64.1 pypi_0 pypi traitlets 5.4.0 pypi_0 pypi twine 3.3.0 pypi_0 pypi typed-ast 1.5.4 pypi_0 pypi typing-extensions 4.3.0 pypi_0 pypi tzdata 2022.2 pypi_0 pypi tzlocal 4.2 pypi_0 pypi uc-micro-py 1.0.1 pypi_0 pypi urllib3 1.26.12 pypi_0 pypi virtualenv 20.16.5 pypi_0 pypi wcwidth 0.2.5 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi websocket-client 1.4.1 pypi_0 pypi werkzeug 2.2.2 pypi_0 pypi wheel 0.37.1 pyhd3eb1b0_0 widgetsnbextension 3.6.1 pypi_0 pypi wrapt 1.14.1 pypi_0 pypi x264 1!157.20191217 h7b6447c_0 xz 5.2.5 h7f8727e_1 zipp 3.8.1 pypi_0 pypi zlib 1.2.12 h5eee18b_3 zstd 1.5.2 ha4553b6_0
Logs ``` Saving config: /home/usr/.sleap/1.2.8/preferences.yaml 2022-09-26 09:31:27.657784: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-09-26 09:31:27.686165: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-09-26 09:31:27.686360: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero Software versions: SLEAP: 1.2.8 TensorFlow: 2.8.3 Numpy: 1.21.5 Python: 3.7.13 OS: Linux-5.15.0-48-generic-x86_64-with-debian-bookworm-sid Happy SLEAPing! :) Segmentation fault (core dumped) ```

Screenshots

How to reproduce

  1. Create new project
  2. Add video
  3. Create skeleton (e.g. two landmarks)
  4. Add instance
  5. Add instance
  6. Right click landmarks until instance disappears
  7. Next action will usually cause a segmentation fault (e.g. adding instance, generating suggested frames)
roomrys commented 1 year ago

First-Pass Analysis

The segmentation/program crashing bug seems to be caused by the hidden user-labeled instances. This issue is also mentioned in #971.

Segmentation Fault

This seems to be caused because we do not add user-labeled instances to the scene in cases where all nodes are hidden. When I change the conditional in https://github.com/talmolab/sleap/blob/develop/sleap/gui/widgets/video.py#L449-L466 such that user-labeled instances are always added to the scene:

if instance.instance.n_visible_points > 0 or not isinstance(instance.instance, PredictedInstance):
            self.view.scene.addItem(instance)

            # connect signal so we can adjust QtNodeLabel positions after zoom
            self.view.updatedViewer.connect(instance.updatePoints)

then the program no longer crashes.

To replicate issue. However, my terminal gives no error message and just exits out of the program:

(sleap_v128) λ sleap-label
Saving config: C:\Users\Liezl/.sleap/1.2.8/preferences.yaml
Restoring GUI state...

Software versions:
SLEAP: 1.2.8
TensorFlow: 2.8.3
Numpy: 1.21.5
Python: 3.7.13
OS: Windows-10-10.0.19041-SP0

Happy SLEAPing! :)

Steps to replicate (same as described by @daskandalis)

  1. New Project
  2. Add video
  3. Create 2 node skeleton, no edges
  4. Add 2 instances on first frame
  5. Right click all nodes on a single instance until instance disappears
  6. Do action (either right click to add a new instance [default, random, average], or generate suggestions [sample] - either lead to issue)

Hidden User-Labeled Instances

User-labeled instances with all nodes set as invisible should still be visible. We currently set instances with all NaNs to be hidden from view. This was originally implemented to hide predicted instances which had all low-scoring nodes (these low-scoring nodes are also denoted by NaNs).

Proposal

We can add an extra conditional that ensures user-labeled instances are always added while PredictedInstances are still subject to a minimum visible node limit.

Relevant Code

https://github.com/talmolab/sleap/blob/develop/sleap/gui/widgets/video.py#L449-L466

daskandalis commented 1 year ago

I replicated this issue in Windows 11 Anaconda Prompt (anaconda3) with the same steps as above:

(sleap) C:\Users\daska\Desktop\PROJECTS\sleap\sleap>sleap-label
Saving config: C:\Users\daska/.sleap/1.2.8/preferences.yaml
2022-09-26 20:23:35.137862: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2022-09-26 20:23:35.138357: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-09-26 20:23:35.141271: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: TOKAMAK-IX
2022-09-26 20:23:35.141412: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: TOKAMAK-IX

Software versions:
SLEAP: 1.2.8
TensorFlow: 2.8.3
Numpy: 1.21.5
Python: 3.7.13
OS: Windows-10-10.0.22000-SP0

Happy SLEAPing! :)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\daska\anaconda3\envs\sleap\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\daska\anaconda3\envs\sleap\lib\multiprocessing\spawn.py", line 113, in _main
    preparation_data = reduction.pickle.load(from_parent)
EOFError: Ran out of input