search

tobac-project / tobac

Tracking and object-based analysis of clouds
BSD 3-Clause "New" or "Revised" License
101 stars 54 forks source link

`test_spectral_filtering` can fail due to random integer #276

Open freemansw1 opened 1 year ago

freemansw1 commented 1 year ago

Apparently test_spectral_filtering can (very rarely) fail if a certain (unknown what the value is) integer is returned by the np.random.randint call.

Apologies for the not very helpful issue- this just happened to me, and I am unable to replicate, but I wanted to note it if it came up again. The error was in this test:

` assert ( abs( np.floor(np.log10(abs(filtered_data.mean())))

and the value on the left hand side was 0.0.

JuliaKukulies commented 1 year ago

Thanks for reporting, @freemansw1- I will have a look at this!

JuliaKukulies commented 1 year ago

I just checked for all possible integers in the specified range (between signal_min and matrix.shape[-1]) if the test fails and I am actually not able to reproduce neither the fail of the test nor any case for which the left hand side equals 0.

I figured that if the matrix in l. 21 is not reinitialized with 0, the test might fail if the data is manipulated more than once. Theoretically this should not happen because the referred line is always called in the test function. Or is there any odd situation where pytest might be interrupted and continues with saved variables such that the test function does not run from the beginning?

freemansw1 commented 1 year ago

Okay, this happened again to me, so I got curious.

It fails (at least) when signal_idx == 4. If you try that and it passes (as long as that is a valid value), that suggests to me that this could be machine-specific, library-specific, or something similar. If it is machine-specific/ISA-specific, this means we get to file a bug upstream!

Info for posterity: CPU: M2 Max, 64 GB variant OS: MacOS Ventura 13.2.1 Conda environment: https://gist.github.com/freemansw1/8386c4a153857c6c8c7f9f79ff16f411

JuliaKukulies commented 1 year ago

OK, wild. Thanks for providing the details of your environment and machine! I tried signal_idx = 4 in a few different conda environments and the tests always pass for me. It may actually not be redundant then to include other operating systems in our CI?

I will think about this a bit more...