strange `casa-task` flavour in-line script not working

[SpheMakh]

Environment

• OS: Ubuntu 22.04
• Python 11
• CASA casa-6.4.1-12-pipeline-2022.2.0.68

I get this error:

### running /home/sphemakh/builds/casa/casa-6.4.1-12-pipeline-2022.2.0.68/bin/casa --log2term --nologger --nologfile -c 
import sys, json
kw = json.loads(sys.argv[-1])

try:
    utype = unicode
except NameError:
    utype = bytes

def stringify(x):
    if isinstance(x, (utype, str)):
        return str(x)
    elif isinstance(x, list):
        return [stringify(y) for y in x]
    else:
        return x

kw = {key: stringify(value) for key, value in kw.items()}

mstransform(**kw)

 {"vis": "msdir/PKS0326-288_VLA-LADUMA_LB.ms", "usewtspectrum": true, "field": "0", "datacolumn": "corrected", "outputvis": "msdir/PKS0326-288_VLA-LADUMA_LB-recipe.mslabel.ms"}
.1# --> CrashReporter initialized.
.1# 2024-05-18 15:01:17 INFO    ::casa  optional configuration file config.py not found, continuing CASA startup without it
.1# 2024-05-18 15:01:17 INFO    ::casa
.1# 2024-05-18 15:01:17 INFO    ::casa  Checking Measures tables in data repository sub-directory /home/sphemakh/builds/casa/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/lib/python3.6/site-packages/casadata/__data__/geodetic
.1# 2024-05-18 15:01:17 INFO    ::casa    IERSeop2000 (version date, last date in table (UTC)): 2022/06/23/15:00, 2022/05/24/00:00:00
.1# 2024-05-18 15:01:17 INFO    ::casa    IERSeop97 (version date, last date in table (UTC)): 2022/06/23/15:00, 2022/05/24/00:00:00
.1# 2024-05-18 15:01:17 INFO    ::casa    IERSpredict (version date, last date in table (UTC)): 2022/06/26/15:00, 2022/09/24/00:00:00
.1# 2024-05-18 15:01:17 INFO    ::casa    TAI_UTC (version date, last date in table (UTC)): 2022/06/20/15:00, 2017/01/01/00:00:00
.1# 
.1# optional configuration file config.py not found, continuing CASA startup without it
.1# 
.1# Python: reductions/vlaredux^GUsing matplotlib backend: TkAgg
.1# CASA 6.4.1.12 -- Common Astronomy Software Applications [6.4.1.12]
.1# /home/sphemakh/builds/casa/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/lib/python3.6/site-packages/IPython/core/interactiveshell.py:935: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
.1#   warn("Attempting to work in a virtualenv. If you encounter problems, please "
.1# Telemetry initialized. Telemetry will send anonymized usage statistics to NRAO.
.1# You can disable telemetry by adding the following line to the config.py file in your rcdir (e.g. ~/.casa/config.py):
.1# telemetry_enabled = False
.1# Traceback (most recent call last):
.1#   File "/home/sphemakh/builds/casa/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/lib/python3.6/site-packages/casashell/private/init_system.py", line 238, in __evprop__
.1#     exec(stmt)
.1#   File "<string>", line 18, in <module>
.1#   File "<string>", line 18, in <dictcomp>
.1# NameError: name 'stringify' is not defined
2024-05-18 15:01:17 STIMELA.selfcal-multi.getdata.getdata ERROR: step 'selfcal-multi.getdata.getdata' has failed, aborting the recipe

The error goes away if I pass the script as a file; see https://github.com/caracal-pipeline/stimela/blob/a2025c7f07ce6a2c4ece6fbabe2aadf93351a3d1/stimela/backends/flavours/casa.py#L73-L99

[o-smirnov]

One of these how-can-this-be problems... the function is right there. How can it be invoking the dict comprehension and not invoking the function definition? Could you please add a "print(stringify)" statement just before the comprehension?

[SpheMakh]

For some ungodly reason, this works

import sys, json
kwin = json.loads('{params_string}')

try:
    utype = unicode
except NameError:
    utype = bytes

kw = dict()

for key, val in kwin.items():
    # stringify in a loop to avoid isue #300
    if isinstance(val, (utype, str)):
        x = str(val)
    elif isinstance(val, list):
        x = [stringify(y) for y in val]
    else:
        x = val

    kw[key] = x

{command}(**kw)

I'll updated #298 accordingly

[SpheMakh]

Oops! there's a big boo boo in the elif clause. Having another go

[o-smirnov]

"Ungodly" indeed! But if it works, who am I to complain...

[SpheMakh]

It doesn't seem to like user-defined variables in nested structures. There's probably some blasphemous code in this this CASA module casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/lib/python3.6/site-packages/casashell/private/init_system.py", line 238, in __evprop__ .1# exec(stmt) (dare to look?)

Anyhoo, here is the updated and working Frankenstein monster

import sys, json
kwin = json.loads('{params_string}')

try:
    utype = unicode
except NameError:
    utype = bytes

kw = dict()

for key, val in kwin.items():
    # stringify in a loop to avoid isue #300
    if isinstance(val, (utype, str)):
        x = str(val)
    elif isinstance(val, list):
        try:
            x = list(map(lambda a: str(a) if isinstance(a, (unicode, str)) else a, val))
        except NameError:
            x = list(map(lambda a: str(a) if isinstance(a, (bytes, str)) else a, val))
    else:
        x = val

    kw[key] = x

{command}(**kw)

[SpheMakh]

Wrapping this up. Ended up using io.StringIO() to read the parameter dictionary and encoding the contents as str or bytes as required by the python version.

https://github.com/caracal-pipeline/stimela/blob/a3775936a56d1cf1dc1dbece8110159955659d1f/stimela/backends/flavours/casa.py#L73-L83