Closed steven-murray closed 1 year ago
steven-murray
commented
2 years ago I should say that this is waiting on https://github.com/RadioAstronomySoftwareGroup/pyuvdata/pull/1205 and https://github.com/RadioAstronomySoftwareGroup/pyuvsim/pull/410 before it will pass tests.
piyanatk
commented
1 year ago This line in hera-sim-vis.py will error when using analytic beams
https://github.com/HERA-Team/hera_sim/blob/03d13da0b921216855e10ac15b36d83013631254/scripts/hera-sim-vis.py#L128
Should I open an issue? It is still in a PR so I am not sure if I should
codecov[bot]
commented
1 year ago Patch coverage: 90.49% and project coverage change: -3.00 :warning:
Comparison is base (
7d860d8) 96.04% compared to head (833c03c) 93.04%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
steven-murray
commented
1 year ago @piyanatk thanks for the review! I've addressed your two comments.
This PR adds quite a lot of functionality/performance improvement.
Fixes #191
Highlights include:
LogRenderandRichHandlerclasses incli_utils.py. These are extensions of theRichHandlerfromrichitself, that add a column for current RAM usage, and also give the option to print out the time since start of script, instead of current absolute time.hera-sim-vis.py._blt_order_kwsclass attribute for theVisibilitySimulatorsubclasses, which specify how to reorder the blt axis of the constructed simulation (if required) before simulating. This is useful as it allow the data to NOT be reordered (which can take some time).run_check_acceptabilitywhen constructing from obsparams (it significantly increases peak memory usage, and is unnecessary since we just made the object from scratch).compress_data_model()method onVisibilitySimulatorsubclasses that allows unnecessary metadata in the UVData object to be dropped before simulation (usually this will have to be restored afterwards with the associatedrestore_data_model()method). This is just to reduce peak memory usage.check_antenna_conjugationparameter for theVisCPUsimulator -- allow turning off the check for antenna conjugation. Usually, doing this is safe, but when constructing the object from obsparams, it really doesn't make sense to do this time-consuming check.VisCPU, ability to not copy the entire data array, but rather just insert data into the existing one. This only triggers if the whole array is made of zeros (i.e. we wouldn't be over-riding data)._reorder_visfor when theblt_orderof the UVData object is('time', 'ant1'). In this case, the output from vis_cpu is already ordered correctly, we just have to take parts of it and stick it in thedata_array. This reduces reordering time by like 99%._reorder_vis, I swapped the loop ordering (for whenblt_orderis not so nice). This halved the time for the reordering.hera-sim-vis.py--compressoption tohera-sim-vis.pyis no longer a boolean flag, but takes a file argument. This file will be written as a cache of the baseline-time indices required to keep when compressing by redundancy. If the file exists, the code will use that cache instead of regenerating the indices, which can itself take a lot of time. This saves many many minutes on H4C-style sims.--log-leveloption tohera-sim-vis.pyto change the verbosity.--dry-runoption tohera-sim-vis.pyto construct the sim config and report on it, but not do the actual sim. Useful for testing setup time and making sure the setup looks correct.Overall, with these improvements, I was able to take a run of 5760 times and 126 antennas from 5min + >30min (for simulation + setup/reorder/write) down to 5min + ~40sec.