UFuncTypeError when running stage1.SuperBiasStep

[hstewart93]

Following the SOSS reduction usage guide, I encounter the following error when running

step = stage1.SuperBiasStep(results, output_dir=outdir_s1)
results = step.run(save_results=True, force_redo=True, do_plot=True)

2024-09-10 16:04:59,631 - stpipe.SuperBiasStep - INFO - SuperBiasStep instance created.
2024-09-10 16:04:59,651 - stpipe.SuperBiasStep - INFO - SuperBiasStep instance created.
2024-09-10 16:04:59,819 - stpipe.SuperBiasStep - INFO - Step SuperBiasStep running with args (<RampModel(100, 14, 256, 2048) from jw02734002001_04101_00001-seg001_nis_saturationstep.fits>,).
2024-09-10 16:04:59,821 - stpipe.SuperBiasStep - INFO - Step SuperBiasStep parameters are:
  pre_hooks: []
  post_hooks: []
  output_file: None
  output_dir: /data/typhon2/hattie/jwst/wasp_96/pipeline_outputs_directory/Stage1/
  output_ext: .fits
  output_use_model: False
  output_use_index: True
  save_results: True
  skip: False
  suffix: None
  search_output_file: True
  input_dir: ''
2024-09-10 16:04:59,858 - stpipe.SuperBiasStep - INFO - Using SUPERBIAS reference file ./crds_cache/references/jwst/niriss/jwst_niriss_superbias_0200.fits

---------------------------------------------------------------------------
UFuncTypeError                            Traceback (most recent call last)
Cell In[28], [line 6]
      [1] step = stage1.SuperBiasStep(results, output_dir=outdir_s1)
      [3] # This step has options for diagnostic plotting. Specifying do_plot=True will create the step
      [4] # plot and save it to the output directory.
----> [6] results = step.run(save_results=True, force_redo=True, do_plot=True)

File ~/.local/share/virtualenvs/jwst-C1x4mLXo/lib/python3.10/site-packages/exotedrf/stage1.py:263, in SuperBiasStep.run(self, save_results, force_redo, do_plot, show_plot, **kwargs)
    [260] # If no output files are detected, run the step.
    [261]else:
    [262]    step = calwebb_detector1.superbias_step.SuperBiasStep()
--> [263]    res = step.call(segment, output_dir=self.output_dir,
    [264]                     save_results=save_results, **kwargs)
    [265]    # Verify that filename is correct.
    [266]    if save_results is True:

File ~/.local/share/virtualenvs/jwst-C1x4mLXo/lib/python3.10/site-packages/stpipe/step.py:686, in Step.call(cls, *args, **kwargs)
    [683]name = config.get("name", None)
    [684]instance = cls.from_config_section(config, name=name, config_file=config_file)
--> [686]return instance.run(*args)

File ~/.local/share/virtualenvs/jwst-C1x4mLXo/lib/python3.10/site-packages/stpipe/step.py:509, in Step.run(self, *args)
    [507]self.prefetch(*args)
    [508] try:
...
     [82]# If ZEROFRAME is present, subtract the super bias.  Zero values
     [83]# indicate bad data, so should be kept zero.
     [84]if input.meta.exposure.zero_frame:

UFuncTypeError: Cannot cast ufunc 'subtract' output from dtype('float32') to dtype('uint16') with casting rule 'same_kind'

Environment is as follows:

Package                   Version
------------------------- ------------------
aiosignal                 1.3.1
anyio                     4.4.0
applesoss                 2.1.0
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asdf                      2.15.2
asdf-astropy              0.6.1
asdf_coordinates_schemas  0.3.0
asdf_standard             1.1.1
asdf_transform_schemas    0.5.0
asdf_unit_schemas         0.2.0
asdf_wcs_schemas          0.4.0
astropy                   6.1.3
astropy-iers-data         0.2024.9.2.0.33.23
astroquery                0.4.7
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     24.2.0
babel                     2.16.0
backports.tarfile         1.2.0
BayesicFitting            3.2.1
beautifulsoup4            4.12.3
bleach                    6.1.0
Bottleneck                1.4.0
certifi                   2024.8.30
cffi                      1.17.1
charset-normalizer        3.3.2
click                     8.1.7
comm                      0.2.2
contourpy                 1.3.0
corner                    2.2.2
crds                      11.18.3
cryptography              43.0.1
cycler                    0.12.1
debugpy                   1.8.5
decorator                 5.1.1
defusedxml                0.7.1
drizzle                   1.14.4
exceptiongroup            1.2.2
executing                 2.1.0
exotedrf                  2.0.0
fastjsonschema            2.20.0
filelock                  3.15.4
fonttools                 4.53.1
fqdn                      1.5.1
frozenlist                1.4.1
future                    1.0.0
gwcs                      0.19.0
h11                       0.14.0
html5lib                  1.1
httpcore                  1.0.5
httpx                     0.27.2
idna                      3.8
importlib_metadata        8.4.0
iniconfig                 2.0.0
ipdb                      0.13.13
ipykernel                 6.29.5
ipython                   8.27.0
ipywidgets                8.1.5
isoduration               20.11.0
jaraco.classes            3.4.0
jaraco.context            6.0.1
jaraco.functools          4.0.2
jedi                      0.19.1
jeepney                   0.8.0
Jinja2                    3.1.4
jmespath                  1.0.1
joblib                    1.4.2
json5                     0.9.25
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2023.12.1
jupyter                   1.1.1
jupyter_client            8.6.2
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.2
jupyter_server_terminals  0.5.3
jupyterlab                4.2.5
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
jupyterlab_widgets        3.0.13
jwst                      1.12.5
keyring                   25.3.0
kiwisolver                1.4.7
MarkupSafe                2.1.5
matplotlib                3.9.2
matplotlib-inline         0.1.7
mistune                   3.0.2
more-itertools            10.4.0
msgpack                   1.0.8
nbclient                  0.10.0
nbconvert                 7.16.4
nbformat                  5.10.4
nest-asyncio              1.6.0
notebook                  7.2.2
notebook_shim             0.2.4
numpy                     1.25.2
opencv-python-headless    4.10.0.84
overrides                 7.7.0
packaging                 24.1
pandas                    2.2.2
pandocfilters             1.5.1
Parsley                   1.3
parso                     0.8.4
pastasoss                 1.1.0
pexpect                   4.9.0
photutils                 1.13.0
pillow                    10.4.0
pip                       24.2
platformdirs              4.2.2
pluggy                    1.5.0
poppy                     1.1.1
prometheus_client         0.20.0
prompt_toolkit            3.0.47
protobuf                  5.28.0
psutil                    6.0.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pycparser                 2.22
pyerfa                    2.0.1.4
Pygments                  2.18.0
pyparsing                 3.1.4
pytest                    8.3.2
python-dateutil           2.9.0.post0
python-json-logger        2.0.7
pytz                      2024.1
pyvo                      1.5.2
PyYAML                    6.0.2
pyzmq                     26.2.0
ray                       2.35.0
referencing               0.35.1
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.20.0
scikit-learn              1.5.1
scipy                     1.9.3
SecretStorage             3.3.3
semantic-version          2.10.0
Send2Trash                1.8.3
setuptools                74.1.2
six                       1.16.0
sniffio                   1.3.1
soupsieve                 2.6
spherical_geometry        1.3.2
stack-data                0.6.3
stcal                     1.4.4
stdatamodels              1.8.4
stpipe                    0.5.2
stsci.image               2.3.9
stsci.imagestats          1.8.3
stsci.stimage             0.2.9
terminado                 0.18.1
threadpoolctl             3.5.0
tinycss2                  1.3.0
tomli                     2.0.1
tornado                   6.4.1
tqdm                      4.66.5
traitlets                 5.14.3
tweakwcs                  0.8.8
types-python-dateutil     2.9.0.20240821
typing_extensions         4.12.2
tzdata                    2024.1
uri-template              1.3.0
urllib3                   2.2.2
wcwidth                   0.2.13
webcolors                 24.8.0
webencodings              0.5.1
websocket-client          1.8.0
wheel                     0.44.0
widgetsnbextension        4.0.13
wiimatch                  0.3.2
zipp                      3.20.1

Python version Python3.10.5, packages installed in virtualenv are:

astropy = "*"
exotedrf = "*"
jupyter = "*"
matplotlib = "*"
ipdb = "*"

Note: This error is also replicated when downloading the WASP39b data as in the tutorial.

[radicamc]

Hi Hattie, yes I've seen this before (actually every time I've run a reduction in the past couple of months). I think there is some backwards compatibility issue when STScI updated some of the reference files. This may potentially be fixed by upgrading to a newer jwst pipeline version (though I have not yet looked into that because upgrading generally introduces a whole new slew of compatibility issues!). In any case, there is a note to this effect in the NIRISS tutorial notebook explaining how to circumnavigate this error. Basically, you just want to create a list of the paths to the saturationstep output files, and then pass that list to the SuperBias step. Something like:

files = [XXXXX_seg001_nis_saturationstep.fits, XXXXX_seg002_nis_saturationstep.fits, ...]
step = stage1.SuperBiasStep(files, output_dir=outdir_s1)
results = step.run(save_results=True, force_redo=True)

For some reason this should work and not give any error, whereas directly passing the data models output from the previous step does.

[hstewart93]

Thanks for this Michael, this work around worked. Sorry I missed the note in the tutorial!