bootstrap tries to modify readonly table

[tikk3r]

Bootstrap crashes for me with the following error:

===================================================== Bootstrap  =====================================================

Running bootstrap
Running: bootstrap.py pipeline.cfg
Running: CleanSHM.py
 - 18:13:23 - ClearSHM                     | Clear shared memory
 - 18:13:23 - ClearSHM                     | Clear Semaphores
 - 18:13:23 - ClearSHM                     | Clear shared dictionaries
For L656064_125MHz_uv_pre-cal.ms changing ref freq from 124218750.0 to 125096130.37109375
Traceback (most recent call last):
  File "/opt/lofar/ddf-pipeline/scripts/bootstrap.py", line 273, in <module>
    run_bootstrap(o)
  File "/opt/lofar/ddf-pipeline/scripts/bootstrap.py", line 88, in run_bootstrap
    m=MSList(o['mslist'])
  File "/opt/lofar/ddf-pipeline/utils/auxcodes.py", line 360, in __init__
    t.putcol('REF_FREQUENCY', freqest1)
  File "/opt/lofar/pyenv-py3/lib64/python3.10/site-packages/casacore/tables/table.py", line 1155, in putcol
    self._putcol(columnname, startrow, nrow, rowincr, value)
RuntimeError: Column REF_FREQUENCY in table /data2/sweijen/rundir_P240+30_ddf-pipeline_no_ateam_clipping/L656064_125MH
z_uv_pre-cal.ms/SPECTRAL_WINDOW is not writable
 - 18:13:24 - ClearSHM                     | Clear shared memory
 - 18:13:24 - ClearSHM                     | Clear Semaphores
 - 18:13:24 - ClearSHM                     | Clear shared dictionaries
FAILED to run bootstrap.py pipeline.cfg: return value is 1
Traceback (most recent call last):
  File "/opt/lofar/ddf-pipeline/scripts/pipeline.py", line 1954, in <module>
    main(o) 
  File "/opt/lofar/ddf-pipeline/scripts/pipeline.py", line 1304, in main
    run('bootstrap.py '+' '.join(sys.argv[1:]),log=None,dryrun=o["dryrun"])
  File "/opt/lofar/ddf-pipeline/utils/auxcodes.py", line 68, in run
    die('FAILED to run '+s+': return value is '+str(retval),database=database)
  File "/opt/lofar/ddf-pipeline/utils/auxcodes.py", line 51, in die
    raise RuntimeError(s)
RuntimeError: FAILED to run bootstrap.py pipeline.cfg: return value is 1

It seems related to auxcodes.py where the table is opened read-only when it tries to update REF_FREQUENCY, causing it to crash: https://github.com/mhardcastle/ddf-pipeline/blob/68afe568914d6b0e86fe085fa2d05ec3b1c1b383/utils/auxcodes.py#L353-L362

[cyriltasse]

We've been discussing that here - https://github.com/mhardcastle/ddf-pipeline/issues/305 . I think you can swict to readonly=False

[mhardcastle]

Did you push a change, Cyril?

---

From: cyriltasse @.> Sent: Wednesday, April 12, 2023 12:44:43 PM To: mhardcastle/ddf-pipeline @.> Cc: Subscribed @.***> Subject: Re: [mhardcastle/ddf-pipeline] bootstrap tries to modify readonly table (Issue #303)

We've been discussing that here - #305https://github.com/mhardcastle/ddf-pipeline/issues/305 . I think you can swict to readonly=False

— Reply to this email directly, view it on GitHubhttps://github.com/mhardcastle/ddf-pipeline/issues/303#issuecomment-1505057130, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABIXE3FO65P56FMYRSTQIZLXA2BRXANCNFSM6AAAAAAWQMXOCA. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[cyriltasse]

I pushed to a different ddf-pipeline branch (ddfFixesApril23 - from which I compile the singularity image) cos I didn't want to mess up too much with your repo... but I can merge or commit to it if you feel it's best

[tikk3r]

Oops, sorry, I should have searched the existing issues better... Changing to readonly=False indeed helped.

[mhardcastle]

Committed a fix to master. I don't think this should be being done in this way but it's OK for now.