v2dl3-vegas: Optimization, ITM Support, FoV Cuts

tmcahill commented 2 years ago

This v2dl3-vegas patch contains optimization and new options for event filling.

This update is not as large as the line count implies. Most is from refactoring EventClass.py and parts of irfloader.py into EffectiveAreaFile.py for the optimization.

New features

-r flag to select reconstruction type; standard (default) or ITM
automatic FoV event cut
-nf flag to disable automatic FoV cutting

Optimization

EffectiveAreaFile objects now wrap all effective area files in v2dl3-vegas. This is necessary for a few reasons:

Reopening the EA ROOT file on every run when doing things such as filling events or IRF was highly inefficient
Grouping run-independent EA file I/O operations to a run-independent object is better object-oriented design
v2dl3-vegas was massively slowed down by redundant EA file I/O

This also makes the code cleaner. I was able to streamline a lot of code where there were previously multiple behaviors for whether or not EA files were being wrapped by an EventClass. Replacing EventClass and associated var names with EffectiveAreaFile is more clear than how I'd previously used event class names in behaviors that weren't exclusive to event class mode.

Runtime

I measured runtime for this optimization using 3 program runs of 12-run runlists. Outputs from the runtime tests were compared with fitsdiff and found to have no differences.

Runtime was 2.5x faster when not using event classes:

Before:
real    0m31.767s  user    0m28.083s  sys     0m3.067s
real    0m31.388s  user    0m28.110s  sys     0m2.768s
real    0m32.592s  user    0m28.396s  sys     0m3.161s

mean real: 31.916

After:
real    0m12.827s  user    0m10.776s  sys     0m1.586s
real    0m12.763s  user    0m10.831s  sys     0m1.479s
real    0m13.173s  user    0m10.926s  sys     0m1.559s

mean real: 12.921

Runtime was 3x faster when using two event classes:

Before:
real    0m54.721s  user    0m49.755s  sys     0m3.886s
real    0m55.445s  user    0m50.119s  sys     0m4.344s
real    0m54.907s  user    0m49.727s  sys     0m4.016s

mean real: 55.024

After:
real    0m19.896s  user    0m15.936s  sys     0m2.041s
real    0m18.791s  user    0m15.812s  sys     0m2.014s
real    0m18.856s  user    0m15.902s  sys     0m1.992s

mean real: 19.181

Our computer cluster yielded very similar results. v2dl3-vegas is now more than twice as fast.

All features from this branch have been independently tested and verified with fitsdiff. Control tests for pre-existing use cases were also ran and compared to main branch outputs.

Changelog

genHDUList.py

New optional args for fov cuts and reco mode
Updated VegasDataSource constructor accordingly

v2dl3_for_vegas.py

Simplified runlist_to_file_pair() now that EffectiveAreaFile wraps all EA files
New flags disabling fov cuts and reco type selection
Simplified runlist mode loop now that EffectiveAreaFile wraps all EA files
var renaming to reflect EffectiveAreaFile updates

EventClass.py -> EffectiveAreaFile.py

Now houses EA operations:
- Vegas/ROOT loading from VegasDataSource.py and irfloader.py
- __build_index__() from irfloader.py
- get_safe_energy() from irfloader.py
Additional cut searches FoVCutUpper, FoVCutLower, and ThetaSquareUpper

VegasDataSource.py:

Positional arg effective_area changed to ea_files
Dev exceptions updated to reflect EffectiveAreaFile wrapper requirement
EA operations moved to EffectiveAreaFile.py
File simplified now that EffectiveAreaFile wraps all EA files
New optional args for various event cuts and reco mode
Pre-existing typo "etv_file" fixed to "evt_file"

fillEVENTS_not_safe.py:

Added ea_files positional argument
New optional args for various event cuts and reco mode
Simplifications now that EffectiveAreaFile wraps all EA files
reco variable now stores ITM or standard reconstruction from the selectedEventsTree
Event averages moved up to consider all events regardless of cut
FoV checks and function added

fillRESPONSE_not_safe.py:

Greatly simplified now that EffectiveAreaFile wraps EA files

irfloader.py:

Greatly simplified now that EffectiveAreaFile wraps EA files
IRFLoader object refactored into EffectiveAreaFile
getIRF() now a standalone function rather than a class function of IRFLoader
- git diff doesn't track the changes to this function well due to the indent level change.
- The only change made was manager and axis_dict load from the provided event_class rather than the previous IRFLoader class.

For VEGAS review

Should be easy, not much was added that wasn't already being done in V2DL3.

fillEVENTS_not_safe.py
- ITM support (lines 82-89)
- FoV exclusion (lines 250-268)
EffectiveAreaFile.py ROOT operations. These may not need much scrutiny as they were simply moved over from irfloader.py

tmcahill commented 2 years ago

Thanks for the quick review Dr. Maier!

tmcahill commented 2 years ago

v2dl3-vegas CI is successful. Fits differences are accounted for:

single-evclass:

Differences in average pointing values come from new change to include events which get cut into the averages.
Difference in number of event rows comes from the new automatic FoV cutting of events
Difference in EFFAREA comes from new average pointing values

point-like-1:

I was not aware that the RAD_MAX was bugged in the main branch. My new way seems to work correctly, as the EA file for point-like-1 has a T^2 upper of 0.01
These don't differ in number of events because the point-like EA files have no FoV cut

point-like-2:

Same as point-like-1, this RAD_MAX is also correct for the point-like-2 EA file which has a T^2 upper of 0.005

multi-evclass:

Difference in number of event rows comes from the new automatic FoV cutting of events

VERITAS-Observatory / V2DL3