Recover keys from original P-frames in `i3_to_json.py`

[mlincett]

Previously i3_to_json.py would fill all the keys required by the realtime JSON generation code with dummy values. This was likely an unintended consequence of running the trigger splitter to create SplitUncleanedInIcePulses, which would result the creation of an empty P-frame. See #5 and #6 for context.

With this PR this behavior changes:

• original P-frames are extracted and indexed by uid
• where available, realtime keys are copied over from the original P-frame, otherwise they are filled with dummy values.

To achieve this, some machinery is introduced for this in the form of the fill_key() and fill_missing_keys() functions. The latter replaces functionality previously implemented in alertify().

Additional changes:

• replacement of deprecated from I3Tray import;
• replacement of icetray.load with import of the corresponding module;
• introduction of a suppress_warnings module as a workaround for IceTray Issue #3112. Why not call warnings.filterwarnings() in the main file? Because flake8 would then complain that all following imports are not at the head of the file and I didn't want to disseminate temporary noqa statements everywhere;
• black-ification of the code (I know we don't like this usually but I really needed some auto-formatting). I can revert it if you think it's better.

[mlincett]

Thanks for this. In lieu of running a separate tray, what about adding an optional argument -p for a user-specified pulse-series, which can be copied into SplitUncleanedInIcePulses? Then we can just run over the original p-frame.

This would mitigate some unintended behavior when there is a clash of uid.

It's a possibility, but how do we establish that the user-specified pulse-series is the correct one / equivalent to SplitUncleanedInIcePulses ? For example I would not know which one to pick and I would rather encode default and fallback options rather than leaving it open.

Then I would still go for an opt-in method to choose which keys to copy and avoid flooding the output file with unnecessary data (also it seems the content of at least a couple of keys is not serializable).

[tianluyuan]

Thanks for this. In lieu of running a separate tray, what about adding an optional argument -p for a user-specified pulse-series, which can be copied into SplitUncleanedInIcePulses? Then we can just run over the original p-frame. This would mitigate some unintended behavior when there is a clash of uid.

It's a possibility, but how do we establish that the user-specified pulse-series is the correct one / equivalent to SplitUncleanedInIcePulses ? For example I would not know which one to pick and I would rather encode default and fallback options rather than leaving it open.

Then I would still go for an opt-in method to choose which keys to copy and avoid flooding the output file with unnecessary data (also it seems the content of at least a couple of keys is not serializable).

Yes that's true, it's not possible ensure the passed pulseseries will match SplitUncleanedInIcePulses. Though it would allow some additional flexibility if users wanted to run scans on different pulses.

I think the opt-in method would work (and already exists to some extent with the --extra). Maybe one way to get around the UID for MC is to hash the InIcePulses, which would be unique for all intents and purposes.

[mlincett]

It's a possibility, but how do we establish that the user-specified pulse-series is the correct one / equivalent to SplitUncleanedInIcePulses ? For example I would not know which one to pick and I would rather encode default and fallback options rather than leaving it open. Then I would still go for an opt-in method to choose which keys to copy and avoid flooding the output file with unnecessary data (also it seems the content of at least a couple of keys is not serializable).

Yes that's true, it's not possible ensure the passed pulseseries will match SplitUncleanedInIcePulses. Though it would allow some additional flexibility if users wanted to run scans on different pulses.

This should be rather a feature for Skymap Scanner :)

I think the opt-in method would work (and already exists to some extent with the --extra). Maybe one way to get around the UID for MC is to hash the InIcePulses, which would be unique for all intents and purposes.

I guess this can be done but we still have the issue of the output filename (I would avoid long hashes in filename if possible).

[ric-evans]

That's a lot of diff. What does it look like if you run black with --skip-string-normalization?

FWIW this repo doesn't have much history anyway, so I think the big diff is okay