OverflowError in pycbc_multi_inspiral script because of numpy 2.x

[Thomas-JACQUOT]

I was running some tests by following this repository and most part of my jobs failed and gave me this error :

Traceback (most recent call last):
  File "/home/thomas.jacquot/.conda/envs/pygrb-cal-test/bin/pycbc_multi_inspiral", line 688, in <module>
    idx_dict = {
               ^
  File "/home/thomas.jacquot/.conda/envs/pygrb-cal-test/bin/pycbc_multi_inspiral", line 693, in <dictcomp>
    idx[ifo]
OverflowError: Python integer -19 out of bounds for uint32

2024-11-25 04:40:48,921:ERROR:pegasus-analyzer(1515): Workflow Failed  wf_uuid: 3222eaeb-e919-40c2-8cea-7957c0a09af6 submit dir: /local/thomas.jacquot/pycbc-tmp_js4yehym/work
2024-11-25 04:40:48,921:ERROR:pegasus-analyzer(2147): One or more workflows failed

I've found out thanks to test that this error is coming from my numpy version (2.1.3) as the following code is showing it :

In [1]: import numpy as np

In [2]: np.uint32(-19)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
Cell In[2], line 1
----> 1 np.uint32(-19)

OverflowError: Python integer -19 out of bounds for uint32
In [3]: exit()
(envtest) jacquot@ll-jacquot:~$ conda list numpy
# packages in environment at /home/jacquot/miniconda3/envs/envtest:
#
# Name                    Version                   Build  Channel
numpy                     2.1.3           py311h08b1b3b_0  
numpy-base                2.1.3           py311hf175353_0 

And by changing the version of numpy the problem does not exist anymore :

In [1]: import numpy as np

In [2]: np.uint32(-19)
<ipython-input-2-5d2cc34e6e70>:1: DeprecationWarning: NumPy will stop allowing conversion of out-of-bound Python integers to integer arrays.  The conversion of -19 to uint32 will fail in the future.
For the old behavior, usually:
    np.array(value).astype(dtype)`
will give the desired result (the cast overflows).
  np.uint32(-19)
Out[2]: 4294967277

In [3]: exit()
(envtest) jacquot@ll-jacquot:~$ conda list numpy
# packages in environment at /home/jacquot/miniconda3/envs/envtest:
#
# Name                    Version                   Build  Channel
numpy                     1.24.3          py311h08b1b3b_1  
numpy-base                1.24.3          py311hf175353_1  

So I think something can be improve in pycbc_multi_inspiral to avoid this error for the future ?

[spxiwh]

@Thomas-JACQUOT I don't this is a numpy version issue. Doing numpy.uint32(-19) is a bug (if one really wants the wraparound, it can be done with % MAX_UINT_32, but I'm pretty sure that 4294967277 is not intended here).

[titodalcanton]

I think the problem is here: https://github.com/gwastro/pycbc/blob/master/bin/pycbc_multi_inspiral#L694-L696. We are doing

idx[ifo] - time_delay_idx[slide][position_index][ifo] < len(snr_dict[ifo])

when in reality we should do

idx[ifo] < time_delay_idx[slide][position_index][ifo] + len(snr_dict[ifo])