search

aperiosoftware / astropy

Repository for the Astropy core package
www.astropy.org
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Investigate occasional segfault in test suite #42

Closed astrofrog closed 1 year ago

astrofrog commented 1 year ago

I still occasionally get a segfault in the test suite so we should probably run the test suite with valgrind again to see if we can identify issues.

Cadair commented 1 year ago

I got the following out of valgrind.

My main question from looking at the code, is just given the input bytes how do we determine how many output elements there are. At the moment we just seem to be dividing the number of bytes by 4?! that seems arbitrary?

==2494703== Invalid write of size 2
==2494703==    at 0x3377B049: pl_p2li (pliocomp.c:106)
==2494703==    by 0x33773E27: compress_plio_1_c (compression.c:121)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e508 is 0 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid write of size 2
==2494703==    at 0x3377B001: pl_p2li (pliocomp.c:153)
==2494703==    by 0x33773E27: compress_plio_1_c (compression.c:121)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e50a is 2 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid write of size 2
==2494703==    at 0x3377AF26: pl_p2li (pliocomp.c:123)
==2494703==    by 0x33773E27: compress_plio_1_c (compression.c:121)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e50c is 4 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid write of size 2
==2494703==    at 0x3377B03B: pl_p2li (pliocomp.c:104)
==2494703==    by 0x33773E27: compress_plio_1_c (compression.c:121)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e50e is 6 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid read of size 8
==2494703==    at 0x484B9BF: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x4A25BA6: PyBytes_FromStringAndSize (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2A7DE: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2B6FD: _Py_BuildValue_SizeT (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x33773E38: compress_plio_1_c (compression.c:125)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e508 is 0 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid read of size 8
==2494703==    at 0x484B9C7: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x4A25BA6: PyBytes_FromStringAndSize (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2A7DE: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2B6FD: _Py_BuildValue_SizeT (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x33773E38: compress_plio_1_c (compression.c:125)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e510 is 8 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703== 
==2494703== Invalid read of size 8
==2494703==    at 0x484B9CF: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x4A25BA6: PyBytes_FromStringAndSize (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2A7DE: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A2B6FD: _Py_BuildValue_SizeT (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x33773E38: compress_plio_1_c (compression.c:125)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==  Address 0x6f5e518 is 16 bytes after a block of size 40 alloc'd
==2494703==    at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2494703==    by 0x33773DE1: compress_plio_1_c (compression.c:111)
==2494703==    by 0x4A74BC0: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x49EF843: _PyObject_MakeTpCall (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4997332: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4A74A9D: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4999BD5: _PyEval_EvalFrameDefault (in /usr/lib/libpython3.11.so.1.0)
==2494703==    by 0x4B80CFF: ??? (in /usr/lib/libpython3.11.so.1.0)
==2494703==
Cadair commented 1 year ago

I think I squashed this.