feature/ga on Linux: coredump

[sanderjo]

sander@haring1:~/git/sabyenc$ tests/speed_compare.py
4.0.1
---
     yEnc C New  took   619 ms
---

sander@haring1:~/git/sabyenc$ python3 setup.py build
sander@haring1:~/git/sabyenc$ cp build/lib*/*so tests/
sander@haring1:~/git/sabyenc$ tests/speed_compare.py
Illegal instruction (core dumped)

Let's see:

sander@haring1:~/git/sabyenc/tests$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sabyenc3
Illegal instruction (core dumped)
sander@haring1:~/git/sabyenc/tests$

So only the import causes the core dump. Not good.

And to be sure: the main repo just works (6 times faster)

sander@haring1:~/git/yenc_poging2$ python3 tests/speed_compare.py
5.0.0
---
     yEnc C New  took   104 ms
---

[Safihre]

@animetosho Do you know how to best debug this? I tried running pytest on GitHub actions, but it succeeds for Ubuntu. ~Additionally, it shows coredumps for macOS unless I change -O3 to -O0.~ Really strange..

EDIT: Fixed the -O3 problem, it was an edit that I made to your code which was not correct!

[sanderjo]

update:

After really installing with sudo python3 setup.py install, I can run speedtest.py. No more coredump.

So the trick with the local cp of the sabyenc.so causes the coredump?

[animetosho]

You can diagnose the problem using gdb by running something like:

gdb python3
run [arguments]

Use the bt command to get a stack trace, when it crashes.
You can also use one of these to show the instruction it halted on, and see if your CPU supports that instruction.

EDIT: Fixed the -O3 problem, it was an edit that I made to your code which was not correct!
So the trick with the local cp of the sabyenc.so causes the coredump?

Neither of these should cause the issue.

[sanderjo]

FWIW / For Google Reference: My dmesg says:

[vr feb 25 15:01:11 2022] traps: python3[742000] trap invalid opcode ip:7ff9a2e7ddbf sp:7ffdf6721c40 error:0 in sabyenc3.cpython-38-x86_64-linux-gnu.so[7ff9a2e69000+1c000]

[vr feb 25 15:28:46 2022] traps: python3[742946] trap invalid opcode ip:7fa2fd689837 sp:7ffc0b0f35f0 error:0 in sabyenc3.cpython-38-x86_64-linux-gnu.so[7fa2fd619000+7f000]

[sanderjo]

Mysterious, but good news:

With the current ga version 056225fde7187cf2ab6030734f3feb4ee698350f the coredump is gone: I can copy the sabyenc ... so into the working directory, and run speedtest safely:

sander@X501A1:~/git/sabyenc$ git branch -a
* (HEAD detached at origin/feature/ga)
  master
  remotes/origin/HEAD -> origin/master
  remotes/origin/feature/ga
  remotes/origin/master
  remotes/origin/sabyenc-python2
  remotes/origin/temp/subprocessww
  remotes/origin/yenc-original
sander@X501A1:~/git/sabyenc$ cp build/lib.linux-x86_64-3.10/sabyenc3.cpython-310-x86_64-linux-gnu.so tests/

sander@X501A1:~/git/sabyenc$ grep -i sse2 tests/sabyenc3.cpython-310-x86_64-linux-gnu.so 
grep: tests/sabyenc3.cpython-310-x86_64-linux-gnu.so: binary file matches

sander@X501A1:~/git/sabyenc$ python3 tests/speed_compare.py 
---
     yEnc C New  took   136 ms
---

[animetosho]

Ah, probably was the CFLAGS array not being cloned.