I am having the following error when calling viz.show() after creating a mesmerize-viz container with a CNMF run with at least 1 empty contour:
thread '<unnamed>' panicked at src/conv.rs:1182:30:
invalid size
stack backtrace:
0: rust_begin_unwind
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5
1: core::panicking::panic_fmt
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14
2: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
3: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
4: wgpuDeviceCreateBindGroup
5: ffi_call_unix64
6: ffi_call_int
7: cdata_call
8: _PyObject_Call
at /usr/local/src/conda/python-3.11.9/Objects/call.c:343:19
9: PyObject_Call
at /usr/local/src/conda/python-3.11.9/Objects/call.c:355:12
10: do_call_core
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:7349:12
11: _PyEval_EvalFrameDefault
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:5376:22
12: _PyEval_EvalFrame
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_ceval.h:73:16
13: _PyEval_Vector
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:6434:24
14: _PyFunction_Vectorcall
at /usr/local/src/conda/python-3.11.9/Objects/call.c:393:16
15: _PyObject_VectorcallTstate
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_call.h:92:11
16: method_vectorcall
at /usr/local/src/conda/python-3.11.9/Objects/classobject.c:67:20
17: _PyObject_VectorcallTstate
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_call.h:92:11
18: context_run
at /usr/local/src/conda/python-3.11.9/Python/context.c:673
19: cfunction_vectorcall_FASTCALL_KEYWORDS
at /usr/local/src/conda/python-3.11.9/Objects/methodobject.c:443:24
20: do_call_core
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:7321:9
21: _PyEval_EvalFrameDefault
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:5376:22
22: _PyEval_EvalFrame
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_ceval.h:73:16
23: _PyEval_Vector
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:6434:24
24: PyEval_EvalCode
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:1148:21
25: builtin_exec_impl
at /usr/local/src/conda/python-3.11.9/Python/bltinmodule.c:1077:17
26: builtin_exec
at /usr/local/src/conda/python-3.11.9/Python/clinic/bltinmodule.c.h:465:20
27: cfunction_vectorcall_FASTCALL_KEYWORDS
at /usr/local/src/conda/python-3.11.9/Objects/methodobject.c:443:24
28: _PyObject_VectorcallTstate
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_call.h:92:11
29: PyObject_Vectorcall
at /usr/local/src/conda/python-3.11.9/Objects/call.c:299:12
30: _PyEval_EvalFrameDefault
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:4769:23
31: _PyEval_EvalFrame
at /usr/local/src/conda/python-3.11.9/Include/internal/pycore_ceval.h:73:16
32: _PyEval_Vector
at /usr/local/src/conda/python-3.11.9/Python/ceval.c:6434:24
33: _PyFunction_Vectorcall
at /usr/local/src/conda/python-3.11.9/Objects/call.c:393:16
34: pymain_run_module
at /usr/local/src/conda/python-3.11.9/Modules/main.c:300:14
35: pymain_run_python
at /usr/local/src/conda/python-3.11.9/Modules/main.c:595:21
36: Py_RunMain
at /usr/local/src/conda/python-3.11.9/Modules/main.c:680:5
37: Py_BytesMain
at /usr/local/src/conda/python-3.11.9/Modules/main.c:734:12
This is a Rust error from wgpu-native. After a lot of digging, I found this: pygfx/pygfx#812. And after setting some breakpoints I found that it was indeed caused by calling gfx.Geometry when the positions argument was an empty array - specifically, when plotting contours.
(Arguably there shouldn't be an empty contour at all (that component should be filtered out?) but it could happen if the threshold when computing contours after the fact is lower than the power threshold when running CNMF. In any case, it shouldn't cause the kernel to crash.)
CaImAn outputs an empty contour as one or more lines of NaNs, but mesmerize filters that out. I found that just removing that line fixes the issue, because apparently while pygfx doesn't like empty arrays, it's fine with an all-NaN array.
Another issue is that sometimes cells have contours with multiple components, separated by NaNs. That's not ideal, but I know that at least when I started exploring CNMF parameters, a lot of my cells were like that. When trying out mesmerize-viz at the time, the contours looked very weird and jagged and I couldn't make heads or tails of them - I think this is why. The individual components were connected by line segments because the NaNs had been removed. I confirmed that after this change, my contours look normal.
I was going to make a PR, but I saw that one of the tests has to be updated with new ground-truth data, so before doing that I just wanted to bring this up and make sure I'm not missing anything.
One more thing is that in the same function, the center of mass is calculated based on the contour (which is different from how it's done in CaImAn). I'm not sure if it's used for anything but should that be consistent? The 'CoM' field of the get_contours output should be accurate (there was a bug if swap_dim was true, but it was just fixed). If that is changed, the ground-truth com data would also have to be changed.
ScreenshotsPlease include screenshots that describe the issue.
To-do if needed
Hardware and environment (please complete the following information):
OS: Windows 10, Linux
mesmerize-core version: master branch
caiman version: dev branch
Hardware description, RAM, CPU, GPU, etc.: tested on multiple machines but can add if needed to replicate
I am having the following error when calling
viz.show()after creating a mesmerize-viz container with a CNMF run with at least 1 empty contour:This is a Rust error from wgpu-native. After a lot of digging, I found this: pygfx/pygfx#812. And after setting some breakpoints I found that it was indeed caused by calling
gfx.Geometrywhen thepositionsargument was an empty array - specifically, when plotting contours.(Arguably there shouldn't be an empty contour at all (that component should be filtered out?) but it could happen if the threshold when computing contours after the fact is lower than the power threshold when running CNMF. In any case, it shouldn't cause the kernel to crash.)
The reason we end up with an empty contour is this line: https://github.com/nel-lab/mesmerize-core/blob/38f6ebebf6dc38bc5f9312751f2848952711807e/mesmerize_core/caiman_extensions/cnmf.py#L306
CaImAn outputs an empty contour as one or more lines of NaNs, but mesmerize filters that out. I found that just removing that line fixes the issue, because apparently while pygfx doesn't like empty arrays, it's fine with an all-NaN array.
Another issue is that sometimes cells have contours with multiple components, separated by NaNs. That's not ideal, but I know that at least when I started exploring CNMF parameters, a lot of my cells were like that. When trying out mesmerize-viz at the time, the contours looked very weird and jagged and I couldn't make heads or tails of them - I think this is why. The individual components were connected by line segments because the NaNs had been removed. I confirmed that after this change, my contours look normal.
I was going to make a PR, but I saw that one of the tests has to be updated with new ground-truth data, so before doing that I just wanted to bring this up and make sure I'm not missing anything.
One more thing is that in the same function, the center of mass is calculated based on the contour (which is different from how it's done in CaImAn). I'm not sure if it's used for anything but should that be consistent? The
'CoM'field of theget_contoursoutput should be accurate (there was a bug ifswap_dimwas true, but it was just fixed). If that is changed, the ground-truth com data would also have to be changed.Screenshots Please include screenshots that describe the issue.
To-do if needed
Hardware and environment (please complete the following information):