Open aeoranday opened 4 months ago
In the standard running sum AVX2 implementation, there is a division by 10 on the sample data that does not get reset. https://github.com/DUNE-DAQ/fdreadoutlibs/blob/1ec993bd73ea455e52dd2b9d0ad186dd2c679956/include/fdreadoutlibs/wibeth/tpg/ProcessStandardRSAVX2.hpp#L137-L144
This further affects the sum and requires an unexpected, low threshold.
This division is necessary for the running sum R factor, so this can be resolved by only dividing by 10 on that part.
In the standard running sum AVX2 implementation, there is a division by 10 on the sample data that does not get reset. https://github.com/DUNE-DAQ/fdreadoutlibs/blob/1ec993bd73ea455e52dd2b9d0ad186dd2c679956/include/fdreadoutlibs/wibeth/tpg/ProcessStandardRSAVX2.hpp#L137-L144
This further affects the sum and requires an unexpected, low threshold.
This division is necessary for the running sum R factor, so this can be resolved by only dividing by 10 on that part.