Stimela binds measurement set as read/write when write permissions are not required

[talonmyburgh]

For stimela recipes not requiring write access to the measurement set, stimela should load the measurement set in read-only mode. An example error is seen below where wsclean is operating with flag no-update-model-required but stimela still attempts to open the measurement set in read/write and the process fails:

[landmanbester]

I don't think stimela is the culprit here. I think this happens because /net is read only

[talonmyburgh]

Yes but that should be fine - we don't need to write to the ms. If you run the wsclean command as is stipulated in the logs, it runs fine. Stimela unnecessarily tries to mount the ms as read/write.

[JSKenyon]

Yeah - I would expect the mounting to fail if stimela was at fault. The only alternative is if wsclean is actually inspecting the permissions on the mounted MS directory and deciding to use write locks based on that information. This should be easy enough to check if you mangle the wsclean cab to have writable: false in its definition of the MS.

[talonmyburgh]

See this comment by @o-smirnov https://chat.google.com/room/AAAAhSZnW6c/jA0EQsmI5kM/oGts36xb9R8?cls=10

[landmanbester]

Ah, ok. Sorry I missed that thread. I'm surprised wsclean runs at all because /net should be read only

[talonmyburgh]

It is read only, but wsclean does not require write permissions (as far as I know) when using the no-update-model-required in the args list above :)

[o-smirnov]

Yeah, it should be a simple fix, just a matter of getting the logic right. I think if (-no-update-model-required is set OR -niter 0) AND -predict is not set AND -store-imaging-weights is not set, then wsclean should not need to write to the MS. We could modify the dynamic schema easily.

But first @talonmyburgh, as per @JSKenyon's suggestion, could you try hardcoding writable: false here and see if it runs for you?

[talonmyburgh]

Did not seem to work, changed the line, reinstalled stimela and cult-cargo but still get this error: Can confirm the quoted wsclean command works for version 2.10.1.

[o-smirnov]

OK so maybe it's a wsclean 3.3+ problem after all. Well, we can ask stimela to try the old one too. Can you add

cab:
  wsclean:
    image:
      version: 2.10.1-cc0.1.2  # assuming your cult-cargo is 0.1.2?

to the top of your recipe (or options) and re-run?

[talonmyburgh]

Where do I do this if I am using breifast.recipes::tron.yml ? I tried adding it to breifast/recipes/tron.yml but got errors doing that... Also does not work to add it to my local-env.yml.

[o-smirnov]

Add local-env.yml to your command-line?

[o-smirnov]

Sorry, cabs: wsclean.

[talonmyburgh]

Still not having luck... maybe I'm misunderstanding what I need to do. local-env.yml:

lib:
  misc:
    breifast:
      # common options for breifast cab can set en masse here
      cabs:
        # use virtual_env breifast for now
        wsclean:
          image:
            version: 2.10.1-cc0.1.3  # assuming your cult-cargo is 0.1.2?
        backend:
          select: native

gives:

$ stimela run breifast.recipes::tron.yml tron obs=47Tuc
2024-09-16 11:45:48 STIMELA INFO: starting                                                                               
2024-09-16 11:45:48 STIMELA INFO: loaded full configuration from cache                                                   
2024-09-16 11:45:48 STIMELA INFO: saving config dependencies to ./stimela.config.deps                                    
2024-09-16 11:45:48 STIMELA INFO: will load recipe/config file                                                           
/home/myburgh/venv/breifast_env/lib/python3.10/site-packages/breifast/recipes/tron.yml                                   
2024-09-16 11:45:48 STIMELA INFO: treating 'tron' as a recipe or cab name                                                
2024-09-16 11:45:50 STIMELA ERROR: error applying configuration from                                                     
/home/myburgh/venv/breifast_env/lib/python3.10/site-packages/breifast/recipes/tron.yml: Key 'wsclean' not in 'Cab'       
    full_key: cabs.breifast.stack-time-cube.wsclean                                                                      
    reference_type=Cab                                                                                                   
    object_type=Cab

[o-smirnov]

That's because wsclean is a top-level cab definition, it's not inside breifast. Add this to your local-env.yml:

cabs:
  wsclean:
    image:
      version: 2.10.1-cc0.1.3  # assuming your cult-cargo is 0.1.2?

[talonmyburgh]

Right sorry, my bad. Can confirm that adding:

cabs:
  wsclean:
    image:
      version: 2.10.1-cc0.1.3  # assuming your cult-cargo is 0.1.2?

to my local-env.yml and no-update-model-required: true to the image-htc step of breifast/breifast-tron/breifast/recipes allows me to run breifast on a measurement set for which I have no write permissions.

[o-smirnov]

OK, great, thanks! That gets us to the bottom of the problem then. Clearly, 2.x wsclean is more careful about opening the MS as read-only when it doesn't need to be writing to it, and somewhere in 3.x it got a bit more careless about that.

I'll report the "bug" to Andre, but first let's check if it's still the case with the latest release (3.5). I have pushed new images -- @talonmyburgh, could you do one final check with version: 3.5-cc0.1.3 please?

[talonmyburgh]

That did not work, failed with this error again: