OutlierDetectionStep producing extraneous outlier_i2d files

[stscijgbot-jp]

Issue JP-3311 was created on JIRA by Jonathan Eisenhamer:

Through a user support call, it has been discovered that OutlierDetectionStep is creating files with the suffix and type of _outlier_i2d. This output does not appear to be documented. The creation of such files does not honor either the save_results or save_intermediate_results parameters. Even when both are False, the files are still created.

Suggestion that, as a minimum, this output be documented. Ideally, the creation should be regulated by a step parameter.

[stscijgbot-jp]

Comment by Tyler Pauly on JIRA:

FWIW, when testing for a Help Desk ticket regarding outlier_detection's "in_memory" flag, I noticed that these files are generated with the default "in_memory=False", but do not appear to be created when "in_memory=True". This was during a run of Spec3Pipeline on NRS_MSASPEC data (jw01810).

The related ticket may indicate that these files hold the intermediate outlier results for each frame, which are held in memory when told to do so rather than dumped to a file.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Makes sense that these intermediate files are being created when in_memory is turned off, but they should get cleaned up (deleted) when the step is done with them, unless the save_intermediate_results param is turned on.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Fixed by #7845

[stscijgbot-jp]

Comment by Melanie Clarke on JIRA:

Testing with v1.12.0 on NIRSpec MOS observation jw02736-o008 in spec3, with --steps.outlier_detection.save_intermediate_results=True, I see 5 kinds of intermediate files written to disk:

jw02736008001_031010000?{}nrs?{_}[0-5]_blot.fits

jw02736008001_03101_0000?_nrs?_blot.fits

jw02736008001_03101_0000?_nrs?_outlier_outlier_s2d.fits.fits

jw02736008001_03101_0000?_nrs?_outlier_i2d.fits

jw02736008001_03101_0000?_nrs?_median.fits

A couple issues:

• These files are overwritten for each source in the observation, so only the last ones are kept on disk. The name appears to be based on the spec2 input file pattern - perhaps it should be based on the output file pattern, incorporating the source name?
• The *outlier__outlier_s2d.fits.fits files appear to be badly named

Also, a minor note – the change log should reference the parameter "save_intermediate_files", not "save_intermediary_files".

Input files were: jw02736008001_03101_0000[2-4]_nrs[1-2]_cal.fits

 

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

I tested this on a set of MIRI Imaging data through calwebb_image3. When I specified pipe3.outlier_detection.save_intermediate_results = True, the following files were left in the directory.

jw01040001005_03103_00001_mirimage_a3001_blot.fits jw01040001005_03103_00002_mirimage_a3001_blot.fits jw01040001005_03103_00003_mirimage_a3001_blot.fits jw01040001005_03103_00004_mirimage_a3001_blot.fits jw01040001005_03103_00005_mirimage_a3001_blot.fits

jw01040001005_03103_00001_mirimage_outlier_i2d.fits jw01040001005_03103_00002_mirimage_outlier_i2d.fits jw01040001005_03103_00003_mirimage_outlier_i2d.fits jw01040001005_03103_00004_mirimage_outlier_i2d.fits jw01040001005_03103_00005_mirimage_outlier_a3001_median.fits jw01040001005_03103_00005_mirimage_outlier_i2d.fits

 

When I commented out that line, once calwebb_image3 was finished, I only had

jw01040001005_03103_00005_mirimage_outlier_a3001_median.fits

left in my directory. This seems to work as intended for imaging.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Misty Cracraft Actually the only file(s) that should be left behind by outlier_detection are the "crf" products. The median is also intermediate and Tyler Pauly is working on removing that one too.

[stscijgbot-jp]

Comment by Misty Cracraft on JIRA:

Yes, but this ticket specifically mentioned blot and outlier_i2d, and both of those vanished. When the other ticket is finished, I'll test it as well and let you know if the median file also vanishes. And yes, I do have crf files in my directory as well.

[stscijgbot-jp]

Comment by Alicia Canipe on JIRA:

Tested this on some NIRCam imaging data and the median, outlier_i1d, and blot images are still getting saved (see log copied below). It's possible I'm doing something dumb, but with just a straightforward call of the step they're getting saved to disk even with save_results=False. 

# Run the step
outlier_detection = OutlierDetectionStep()
outlier_detection.save_results = False
outlier = outlier_detection.call(sky)```
```java
2023-10-02 10:33:47,923 - stpipe.OutlierDetectionStep - INFO - OutlierDetectionStep instance created.
2023-10-02 10:33:47,943 - stpipe.OutlierDetectionStep - INFO - OutlierDetectionStep instance created.
2023-10-02 10:33:48,072 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep running with args (<ModelContainer>,).
2023-10-02 10:33:48,073 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': False, 'input_dir': '', 'weight_type': 'ivm', 'pixfrac': 1.0, 'kernel': 'square', 'fillval': 'INDEF', 'nlow': 0, 'nhigh': 0, 'maskpt': 0.7, 'grow': 1, 'snr': '5.0 4.0', 'scale': '1.2 0.7', 'backg': 0.0, 'kernel_size': '7 7', 'threshold_percent': 99.8, 'ifu_second_check': False, 'save_intermediate_results': False, 'resample_data': True, 'good_bits': '~DO_NOT_USE', 'scale_detection': False, 'allowed_memory': None, 'in_memory': False}
2023-10-02 10:33:48,079 - stpipe.OutlierDetectionStep - INFO - Performing outlier detection on 6 inputs
2023-10-02 10:33:48,079 - stpipe.OutlierDetectionStep - INFO - Driz parameter kernel: square
2023-10-02 10:33:48,080 - stpipe.OutlierDetectionStep - INFO - Driz parameter pixfrac: 1.0
2023-10-02 10:33:48,080 - stpipe.OutlierDetectionStep - INFO - Driz parameter fillval: INDEF
2023-10-02 10:33:48,080 - stpipe.OutlierDetectionStep - INFO - Driz parameter weight_type: ivm
2023-10-02 10:33:48,081 - stpipe.OutlierDetectionStep - INFO - Output pixel scale ratio: 1.0
2023-10-02 10:33:48,211 - stpipe.OutlierDetectionStep - INFO - 2 exposures to drizzle together
2023-10-02 10:33:49,006 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:33:50,947 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:33:52,245 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_0_o101_outlier_i2d.fits
2023-10-02 10:33:52,253 - stpipe.OutlierDetectionStep - INFO - 2 exposures to drizzle together
2023-10-02 10:33:53,307 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:33:55,518 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:33:56,785 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_2_o101_outlier_i2d.fits
2023-10-02 10:33:56,793 - stpipe.OutlierDetectionStep - INFO - 2 exposures to drizzle together
2023-10-02 10:33:57,834 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:34:00,044 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5246)
2023-10-02 10:34:01,336 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_4_o101_outlier_i2d.fits
2023-10-02 10:34:01,430 - stpipe.OutlierDetectionStep - INFO - Computing median
2023-10-02 10:34:13,953 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_0_o101_outlier_o101_median.fits
2023-10-02 10:34:13,956 - stpipe.OutlierDetectionStep - INFO - Blotting median
2023-10-02 10:34:14,791 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:14,952 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_0_o101_o101_blot.fits
2023-10-02 10:34:15,796 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:16,068 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_1_o101_o101_blot.fits
2023-10-02 10:34:17,170 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:17,344 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_2_o101_o101_blot.fits
2023-10-02 10:34:18,443 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:18,631 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_3_o101_o101_blot.fits
2023-10-02 10:34:19,729 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:19,910 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_4_o101_o101_blot.fits
2023-10-02 10:34:21,007 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5246)
2023-10-02 10:34:21,187 - stpipe.OutlierDetectionStep - INFO - Saved model in step_SkyMatchStep_5_o101_o101_blot.fits
2023-10-02 10:34:21,188 - stpipe.OutlierDetectionStep - INFO - Flagging outliers
2023-10-02 10:34:21,384 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 0 (0.00%)
2023-10-02 10:34:21,580 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 1 (0.00%)
2023-10-02 10:34:21,791 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 0 (0.00%)
2023-10-02 10:34:22,010 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 0 (0.00%)
2023-10-02 10:34:22,227 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 0 (0.00%)
2023-10-02 10:34:22,443 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 0 (0.00%)
2023-10-02 10:34:22,455 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep done

 

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Alicia Canipe Hmmm, when we run the full image3/spec3 pipeline we're no longer seeing those files left behind. The default values of the params save_intermediate_results and in_memory are both False, which means 1) the step will create those intermediate files, because it's been told to not keep everything in memory, but 2) it should then get rid of them at the end of processing, because save_intermediate_results=False. 

Can you check the values of those params in your run?

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Alicia Canipe Also, yes it's expected that those files will get created during processing and hence they'll show up in the processing log, but the real question is whether they're still there once the step has completed?

[stscijgbot-jp]

Comment by Alicia Canipe on JIRA:

Yes, sorry – they were getting saved to disk and not being deleted after the step was done running. I was just using default specs, and also tried save_results=False. (Note: this is with version 1.12.0, if that makes any difference)

print(OutlierDetectionStep.spec) ```
 
```java
weight_type = option('ivm','exptime',default='ivm')
pixfrac = float(default=1.0)
kernel = string(default='square') # drizzle kernel
fillval = string(default='INDEF')
nlow = integer(default=0)
nhigh = integer(default=0)
maskpt = float(default=0.7)
grow = integer(default=1)
snr = string(default='5.0 4.0')
scale = string(default='1.2 0.7')
backg = float(default=0.0)
kernel_size = string(default='7 7')
threshold_percent = float(default=99.8)
ifu_second_check = boolean(default=False)
save_intermediate_results = boolean(default=False)
resample_data = boolean(default=True)
good_bits = string(default="~DO_NOT_USE")  # DQ flags to allow
scale_detection = boolean(default=False)
search_output_file = boolean(default=False)
allowed_memory = float(default=None)  # Fraction of memory to use for the combined image
in_memory = boolean(default=False) ```
I was also worried that it was some weird issue with my notebook, so I tried on the command line and still see the extra files:
```java
strun outlier_detection jw01076-o101_20230927t130753_image3_00001_asn.json --save_intermediate_results=False --in_memory=False ```
```java
2023-10-02 15:00:21,402 - stpipe.OutlierDetectionStep - INFO - OutlierDetectionStep instance created.
2023-10-02 15:00:21,435 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep running with args ('█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
2023-10-02 15:00:21,436 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': True, 'skip': False, 'suffix': None, 'search_output_file': False, 'input_dir': '████████████████████████████████████████████████████████████████ 'weight_type': 'ivm', 'pixfrac': 1.0, 'kernel': 'square', 'fillval': 'INDEF', 'nlow': 0, 'nhigh': 0, 'maskpt': 0.7, 'grow': 1, 'snr': '5.0 4.0', 'scale': '1.2 0.7', 'backg': 0.0, 'kernel_size': '7 7', 'threshold_percent': 99.8, 'ifu_second_check': False, 'save_intermediate_results': False, 'resample_data': True, 'good_bits': '~DO_NOT_USE', 'scale_detection': False, 'allowed_memory': None, 'in_memory': False}
...
...
...
2023-10-02 15:00:31,319 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5247)
2023-10-02 15:00:32,613 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00001_nrcalong_outlier_i2d.fits
2023-10-02 15:00:32,628 - stpipe.OutlierDetectionStep - INFO - 2 exposures to drizzle together
2023-10-02 15:00:33,445 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5247)
2023-10-02 15:00:35,448 - stpipe.OutlierDetectionStep - INFO - Drizzling (2048, 2048) --> (2245, 5247)
2023-10-02 15:00:36,745 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00002_nrcalong_outlier_i2d.fits
2023-10-02 15:00:36,867 - stpipe.OutlierDetectionStep - INFO - Computing median
2023-10-02 15:00:49,615 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02102_00001_nrcalong_outlier_o101_median.fits
2023-10-02 15:00:49,618 - stpipe.OutlierDetectionStep - INFO - Blotting median
2023-10-02 15:00:50,561 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:50,713 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02102_00001_nrcalong_o101_blot.fits
2023-10-02 15:00:51,645 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:51,799 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02102_00001_nrcblong_o101_blot.fits
2023-10-02 15:00:52,788 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:52,942 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00001_nrcalong_o101_blot.fits
2023-10-02 15:00:53,867 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:54,023 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00001_nrcblong_o101_blot.fits
2023-10-02 15:00:54,952 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:55,106 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00002_nrcalong_o101_blot.fits
2023-10-02 15:00:56,210 - stpipe.OutlierDetectionStep - INFO - Blotting (2048, 2048) <-- (2245, 5247)
2023-10-02 15:00:56,364 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00002_nrcblong_o101_blot.fits
2023-10-02 15:00:56,365 - stpipe.OutlierDetectionStep - INFO - Flagging outliers
2023-10-02 15:00:56,749 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 16663 (0.40%)
2023-10-02 15:00:57,060 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 19104 (0.46%)
2023-10-02 15:00:57,376 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 16572 (0.40%)
2023-10-02 15:00:57,716 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 18347 (0.44%)
2023-10-02 15:00:58,058 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 17584 (0.42%)
2023-10-02 15:00:58,322 - stpipe.OutlierDetectionStep - INFO - New pixels flagged as outliers: 20624 (0.49%)
2023-10-02 15:00:59,601 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02102_00001_nrcalong_o101_outlierdetectionstep.fits
2023-10-02 15:00:59,790 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02102_00001_nrcblong_o101_outlierdetectionstep.fits
2023-10-02 15:00:59,978 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00001_nrcalong_o101_outlierdetectionstep.fits
2023-10-02 15:01:00,167 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00001_nrcblong_o101_outlierdetectionstep.fits
2023-10-02 15:01:00,354 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00002_nrcalong_o101_outlierdetectionstep.fits
2023-10-02 15:01:00,538 - stpipe.OutlierDetectionStep - INFO - Saved model in jw01076101001_02104_00002_nrcblong_o101_outlierdetectionstep.fits
2023-10-02 15:01:00,539 - stpipe.OutlierDetectionStep - INFO - Step OutlierDetectionStep done
(jwst_1.12.0) jupiter:jwst_image3_nircam_test acanipe$ lsl
total 5699888
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 10:18 jw01076101001_02102_00001_nrcalong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 10:18 jw01076101001_02102_00001_nrcblong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  117547200 Oct  2 10:18 jw01076101001_02104_00001_nrcalong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  117547200 Oct  2 10:18 jw01076101001_02104_00001_nrcblong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  117547200 Oct  2 10:18 jw01076101001_02104_00002_nrcalong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  117547200 Oct  2 10:18 jw01076101001_02104_00002_nrcblong_tweakregstep.fits
-rw-r--r--  1 acanipe  staff  329970240 Oct  2 13:11 step_ResampleStep_resamplestep.fits
-rw-r--r--  1 acanipe  staff    1157813 Oct  2 14:54 jwst_image3_nircam_test.ipynb
-rw-r--r--  1 acanipe  staff  141425280 Oct  2 15:00 jw01076101001_02102_00001_nrcalong_outlier_i2d.fits
-rw-r--r--  1 acanipe  staff  141425280 Oct  2 15:00 jw01076101001_02104_00001_nrcalong_outlier_i2d.fits
-rw-r--r--  1 acanipe  staff  141425280 Oct  2 15:00 jw01076101001_02104_00002_nrcalong_outlier_i2d.fits
-rw-r--r--  1 acanipe  staff   47180160 Oct  2 15:00 jw01076101001_02102_00001_nrcalong_outlier_o101_median.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02102_00001_nrcalong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02102_00001_nrcblong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02104_00001_nrcalong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02104_00001_nrcblong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02104_00002_nrcalong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02104_00002_nrcblong_o101_blot.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02102_00001_nrcalong_o101_outlierdetectionstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02102_00001_nrcblong_o101_outlierdetectionstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:00 jw01076101001_02104_00001_nrcalong_o101_outlierdetectionstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:01 jw01076101001_02104_00001_nrcblong_o101_outlierdetectionstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:01 jw01076101001_02104_00002_nrcalong_o101_outlierdetectionstep.fits
-rw-r--r--  1 acanipe  staff  117538560 Oct  2 15:01 jw01076101001_02104_00002_nrcblong_o101_outlierdetectionstep.fits
(jwst_1.12.0) jupiter:jwst_image3_nircam_test acanipe$  ```
 

[stscijgbot-jp]

Comment by David Law on JIRA:

Just tested this quickly with the latest pipeline and can confirm Alicia Canipe 's issue above is still a problem.

This issue only shows up when running outlier_detection outside of calwebb_image3, and I see it for both NIRCam and MIRI data.  The root issue seems to be that, when running within image3, line 63 of calwebb_image3.py sets the oulier detection suffix to 'crf'.  The code within outlier_detection_step that does the intermediate file cleanup then tries to replace 'crf' in output names with 'outlier_i2d', 'blot' and 'median' to remove those files.

However, when running standalone, it's not writing to 'crf.fits' files, but to 'outlierdetectionstep.fits' files instead.  As a result, the string matching fails to find any 'crf' values in the name to replace, and can't clean up any intermediate files.

Either the cleanup code needs to take this into account or (perhaps better) the output of Outlier Detection should have 'crf' suffixes regardless of whether the step is run standalone or as part of image3.

Reopening.

[stscijgbot-jp]

Comment by Howard Bushouse on JIRA:

Additional fixes/changes contained in https://github.com/spacetelescope/jwst/pull/8464, which is included in DMS B11.0.

[stscijgbot-jp]

Comment by David Law on JIRA:

Looks like this is working, and not producing outlier_i2d files after the latest fix.  Closing.