AlarmInterrupt and bkz_reduction: SystemError bkz_reduction returned NULL without setting an error

[J08nY]

I am using the threading module to run multiple threads of BKZ and CVP solving (on different matrix objects) and the cysignals alarm to set a timeout on the threads to only run BKZ for a given time, I am getting the following error when the timeout is hit:

------------------------------------------------------------------------
/usr/lib/python3.7/site-packages/cysignals/signals.cpython-37m-x86_64-linux-gnu.so(+0x748d)[0x7fe8cb71848d]
/usr/lib/python3.7/site-packages/cysignals/signals.cpython-37m-x86_64-linux-gnu.so(+0x767b)[0x7fe8cb71867b]
/usr/lib/python3.7/site-packages/cysignals/signals.cpython-37m-x86_64-linux-gnu.so(+0x9fa2)[0x7fe8cb71afa2]
/usr/lib/libpthread.so.0(+0x123c0)[0x7fe8cc7ae3c0]
/usr/lib/libfplll.so.5(_ZN5fplll7NumVectINS_4Z_NRIA1_12__mpz_structEEE9addmul_siERKS5_l+0x0)[0x7fe8cb23fc80]
/usr/lib/libfplll.so.5(_ZN5fplll6MatGSOINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE13row_addmul_siEiil+0x117)[0x7fe8cb262f57]
/usr/lib/libfplll.so.5(_ZN5fplll12LLLReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE5babaiEiii+0x2f1)[0x7fe8cb19adc1]
/usr/lib/libfplll.so.5(_ZN5fplll12LLLReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE14size_reductionEiii+0x7c)[0x7fe8cb1abd6c]
/usr/lib/libfplll.so.5(_ZN5fplll12BKZReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE13svp_reductionEiiRKNS_8BKZParamEb+0x6cc)[0x7fe8cb1ecdac]
Exception in thread 111:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "./bkz.py", line 58, in cvp
    reduced = BKZ.reduction(reduced, BKZ.Param(block_size=block_size, strategies=BKZ.DEFAULT_STRATEGY, auto_abort=True))
SystemError: <cyfunction bkz_reduction at 0x7fe8c977f830> returned NULL without setting an error

/usr/lib/libfplll.so.5(_ZN5fplll12BKZReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE3hkzERiRKNS_8BKZParamEii+0x4f)[0x7fe8cb1ec10f]
/usr/lib/libfplll.so.5(_ZN5fplll12BKZReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE4tourEiRiRKNS_8BKZParamEii+0x6a)[0x7fe8cb1ec3ba]
/usr/lib/libfplll.so.5(_ZN5fplll12BKZReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE17svp_preprocessingEijRKNS_8BKZParamE+0x10f)[0x7fe8cb1ec60f]
/usr/lib/libfplll.so.5(_ZN5fplll12BKZReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE13svp_reductionEiiRKNS_8BKZParamEb+0x1ad)[0x7fe8cb1ec88d]
/usr/lib/libc.so.6(_IO_vfprintf+0x7a3)[0x7fe8cc628ef3]
------------------------------------------------------------------------
*** stack smashing detected ***: <unknown> terminated

[malb]

Looks like we're not threadsafe

[malb]

Can you provide a minimal failing example?

[J08nY]

The following code crashes on fpylll 0.5.2dev with fplll 5.3.3. It reaches the BKZ reduction line and then when the alarm fires it SIGSEGVs or similar.

#!/usr/bin/env python3
from threading import Thread
from cysignals.alarm import alarm, AlarmInterrupt
from fpylll import LLL, BKZ, IntegerMatrix

class Test(Thread):

    def __init__(self, matrix):
        super().__init__()
        self.matrix = matrix

    def run(self):
        try:
            print("start")
            alarm(10)
            print("BKZ")
            reduced = BKZ.reduction(self.matrix, BKZ.Param(block_size=40, strategies=BKZ.DEFAULT_STRATEGY, auto_abort=True))
            print("BKZ done")
        except AlarmInterrupt:
            print("Alarm")
            pass

if __name__ == "__main__":
    A = IntegerMatrix(120, 120)
    A.randomize("ntrulike", bits=64)
    t = Test(A)
    t.start()
    t.join()

Output:

start
BKZ
------------------------------------------------------------------------
/usr/lib/python3.8/site-packages/cysignals/signals.cpython-38-x86_64-linux-gnu.so(+0x7dad)[0x7fd13b50ddad]
/usr/lib/python3.8/site-packages/cysignals/signals.cpython-38-x86_64-linux-gnu.so(+0x7e6c)[0x7fd13b50de6c]
/usr/lib/python3.8/site-packages/cysignals/signals.cpython-38-x86_64-linux-gnu.so(+0xa670)[0x7fd13b510670]
/usr/lib/libc.so.6(+0x3d6a0)[0x7fd13be806a0]
/usr/lib/libfplll.so.6(_ZN5fplll6MatGSOINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE13row_addmul_siEiil+0x2d)[0x7fd13a34818d]
/usr/lib/libfplll.so.6(_ZN5fplll12LLLReductionINS_4Z_NRIA1_12__mpz_structEENS_5FP_NRIdEEE5babaiEiii+0x2ed)[0x7fd13a25aa7d]
[0x7fcf49d442b0]
------------------------------------------------------------------------
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "./example.py", line 18, in run
    reduced = BKZ.reduction(self.matrix, BKZ.Param(block_size=40, strategies=BKZ.DEFAULT_STRATEGY, auto_abort=True))
SystemError: <cyfunction bkz_reduction at 0x7fcf49f01c70> returned NULL without setting an error

[malb]

Thanks! This means interrupting the programme while running and then expecting everything to be fine won't work. That is, I was wrong above: this is not a question of thread safety but cold interrupting a computation and expecting a sane state after.

I see two options:

1. use multiprocessing i.e. forking
2. use max_time