search

XENONnT / straxen

Streaming analysis for XENON
BSD 3-Clause "New" or "Revised" License
21 stars 32 forks source link

Update NaN filtering in InterpolatingMap #1302

Closed JelleAalbers closed 8 months ago

JelleAalbers commented 8 months ago

What does the code in this PR do / what does it improve?

Fixes InterpolatingMap so we do not crash with scipy 1.11.

Can you briefly describe how it works?

Scipy 1.11's kdtree no longer accepts nonfinite values (https://github.com/scipy/scipy/pull/18502). However, we sometimes use NaN to represent missing/undefined values, e.g. for drift time if an event has no S1.

Currently we pass all points to InterpolatingMap, then force the result to NaN for points that have nonfinite distance to the neighbours the kdtree finds. That's fine in our current scipy 1.10 environment, but once we update (https://github.com/XENONnT/ax_env/pull/353) there will be problems.

This PR instead checks for NaN before passing the points to InterpolatingMap to avoid crashing.

Can you give a minimal working example (or illustrate with a figure)?

See the new unit test.

I tried processing a small run 026195 locally with scipy 1.11.3, and it crashed here https://github.com/XENONnT/straxen/blob/933cc547dcd0931252f606d5c1692f36b367cc48/straxen/plugins/events/event_positions.py#L164 where events with NaN drift time are being passed to a map. Now it no longer crashes.

dachengx commented 8 months ago

Thanks @JelleAalbers . Just for my education. How did you find this potential future crash?

JelleAalbers commented 8 months ago

Thanks Dacheng -- I was processing a tiny run on my laptop, which meant all the package versions were different from the ones we usually use.