search

ovro-eovsa / ovro-lwa-solar-ops

Scripts and codes related to operations of OVRO-LWA for solar studies
MIT License
1 stars 4 forks source link

Implement near realtime synoptic imaging for fast visibilities #23

Closed binchensun closed 4 months ago

binchensun commented 5 months ago

The images will be made at a cadence of ~1 min (similar to the slow visibility synoptic images). This will be used as a monitoring point for the quality of the fast visibility data.

surajit4444mondal commented 5 months ago

@binchensun @peijin94 Method to test the new realtime pipeline:

  1. Activate /opt/devel/msurajit/envs/my_suncasa_env/ using conda
  2. The environment has updated ovro-lwa-solar which is needed to work with the modified pipeline.

The new pipeline has the following new features:

  1. Do/not do selfcal depending on presence of --do_selfcal.
  2. If --do_selfcal = False, nearest available selfcal tables are applied
  3. Do/not do imaging depending on presence of --do_imaging
  4. Provide which frequency bands on which to run analysis. Is provided using --bands 41MHz 78 MHz
  5. Instead of deleting caltables at each round, only the selfcal caltables for previous time is deleted. So we always have selfcal tables for all frequency bands at any given time.
  6. Can handle both fast and slow vis data. Ensure that the caltable containing bandpass tables are writable if analysing fast_vis data. This is needed because new bandpass tables suitable for fast_vis is created at runtime. We may later choose this to be part of the caltable generation process. In that case, this folder need not be writable.
  7. I advise that we do not do selfcal using fast vis data. Fast vis data now only produces band-averaged images.

For selfcal and imaging in 3 bands for slow visibility run the following code:

python3 /data07/msurajit/ovro-lwa-solar-ops/pipeline_runner.py 2024-04-29T20:42 --end_time 2024-04-29T20:43 --interval 300 --nodes 4 --singlenode  --delay 2 --proc_dir /fast/msurajit/realtime_pipeline/ --save_dir /fast/msurajit/realtime_pipeline/ --calib_file 20240417_100407  --logger_dir /fast/msurajit/realtime_pipeline/  --file_path slow  --calib_dir /fast/msurajit/realtime_pipeline/caltables/ --bands 41MHz 78MHz 55MHz

For only imaging in all bands for slow visibility run the following code:

python3 /data07/msurajit/ovro-lwa-solar-ops/pipeline_runner.py 2024-04-29T20:42 --end_time 2024-04-29T20:43 --interval 300 --nodes 4 --singlenode  --delay 2 --proc_dir /fast/msurajit/realtime_pipeline/ --save_dir /fast/msurajit/realtime_pipeline/ --calib_file 20240417_100407  --logger_dir /fast/msurajit/realtime_pipeline/  --file_path slow  --calib_dir /fast/msurajit/realtime_pipeline/caltables/ --no_selfcal

For only selfcal in all bands for slow visibility run the following code:

python3 /data07/msurajit/ovro-lwa-solar-ops/pipeline_runner.py 2024-04-29T20:42 --end_time 2024-04-29T20:43 --interval 300 --nodes 4 --singlenode  --delay 2 --proc_dir /fast/msurajit/realtime_pipeline/ --save_dir /fast/msurajit/realtime_pipeline/ --calib_file 20240417_100407  --logger_dir /fast/msurajit/realtime_pipeline/  --file_path slow  --calib_dir /fast/msurajit/realtime_pipeline/caltables/ --no_imaging

For only imaging in all bands for fast visibility run the following code:

python3 /data07/msurajit/ovro-lwa-solar-ops/pipeline_runner.py 2024-04-29T20:42 --end_time 2024-04-29T20:43 --interval 300 --nodes 4 --singlenode  --delay 2 --proc_dir /fast/msurajit/realtime_pipeline/ --save_dir /fast/msurajit/realtime_pipeline/ --calib_file 20240417_100407  --logger_dir /fast/msurajit/realtime_pipeline/  --file_path fast  --calib_dir /fast/msurajit/realtime_pipeline/caltables/ --no_selfcal
surajit4444mondal commented 4 months ago

Significant process on this has been made in #21 #26 and #29

binchensun commented 4 months ago

Another major update in PR #30

surajit4444mondal commented 4 months ago

Closing this issue as the fast vis pipeline is working fine.