caracal-pipeline / caracal

Containerized Automated Radio Astronomy Calibration (CARACal) pipeline
GNU General Public License v2.0
28 stars 6 forks source link

unable to set obsconf:obsinfo:plotelev:enable:false #1537

Open pjmac1105 opened 10 months ago

pjmac1105 commented 10 months ago

hi team,

I am trying to stop CARACal running the elevation plots, setting obsconf:obsinfo:plotelev to false, but it will not take the command - I get the following error:

obsconf:
    - Cannot find required key 'enable'. Path: '/obsconf/obsinfo'

Commands were taken from https://caracal.readthedocs.io/en/latest/manual/workers/obsconf/

I copied a different plot into the obsinfo folder and changed the name to lbv2_5500-elevation-tracks.png, but it still tries to run plotelev even though the file exists. That error below.

Not sure what to do? thanks! Peter

job started at 2023-09-12 09:12:01.397796
Starting container [elevation_plots_ms0-14038047218270416945027213380814]. Timeout set to -1. The container ID is printed below.
running cd /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 && singularity run --workdir /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 --containall  --bind /scratch3/projry
WARNING: Overriding HOME environment variable with SINGULARITYENV_HOME is not permitted
2023-09-12 09:12:06: Initial memory state:
2023-09-12 09:12:06:               total        used        free      shared  buff/cache   available
2023-09-12 09:12:06: Mem:           236G         10G        174G        2.9M         51G        224G
2023-09-12 09:12:06: Swap:            0B          0B          0B
2023-09-12 09:12:06: Running plot-elevation-tracks.py --output-name /stimela_mount/output/lbv2_5500-elevation-tracks.png /stimela_mount/msdir/lbv2_5500.ms
Fields in /stimela_mount/msdir/lbv2_5500.ms:
Traceback (most recent call last):
  File "/usr/local/bin/plot-elevation-tracks.py", line 60, in <module>
    field_time, field_radec, obs_xyz, scans = LSTElevationPlot.load_ms_fields(msname, verbose=1)
  File "/usr/local/lib/python3.6/dist-packages/Owlcat/Plotting.py", line 1167, in load_ms_fields
    time_ts = np.array([timestamps[row0] for row0 in first_row_per_ts])        # time per timestamp
  File "/usr/local/lib/python3.6/dist-packages/Owlcat/Plotting.py", line 1167, in <listcomp>
    time_ts = np.array([timestamps[row0] for row0 in first_row_per_ts])        # time per timestamp
IndexError: index 121304 is out of bounds for axis 0 with size 121304
2023-09-12 09:12:12: plot-elevation-tracks.py exited with code 1
cd /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 && singularity run --workdir /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 --containall returns error code 1
job failed at 2023-09-12 09:12:12.311261 after 0:00:10.913465
Traceback (most recent call last):
  File "/scratch3/projects/orcs/orc23_caracal_2/venv_orc23/lib/python3.9/site-packages/stimela/recipe.py", line 713, in run
    job.run_job()
  File "/scratch3/projects/orcs/orc23_caracal_2/venv_orc23/lib/python3.9/site-packages/stimela/recipe.py", line 425, in run_job
    self.job.run(output_wrangler=self.apply_output_wranglers)
  File "/scratch3/projects/orcs/orc23_caracal_2/venv_orc23/lib/python3.9/site-packages/stimela/singularity.py", line 123, in run
    utils.xrun(f"cd {self.execdir} && singularity run --workdir {self.execdir} --containall",
  File "/scratch3/projects/orcs/orc23_caracal_2/venv_orc23/lib/python3.9/site-packages/stimela/utils/xrun_poll.py", line 227, in xrun
    raise StimelaCabRuntimeError("{} returns error code {}".format(command_name, status))
stimela.utils.StimelaCabRuntimeError: cd /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 && singularity run --workdir /scratch3/projects/orcs/zac/5500/.stimela_workdir-1694502721323329 --coni1
log-obsconf-elevation-plots-ms0-20230912-091156.txt (END)
o-smirnov commented 10 months ago

Have you tried setting obsconf: obsinfo: plotelev: enable: False?

I'm also curious why the plotter fails like that in the first place. Is there any way I can get access to your MS?

Athanaseus commented 3 weeks ago

@pjmac1105 please re-open if still experiencing the issue.