JP-3746: Improve background combination for NIRSpec MOS master background

[hayescr]

Resolves JP-3746

Closes #8788

This PR adds an option to do a pixel-by-pixel "sigma clipping" with a median and median absolute deviation on MOS background spectra to remove outliers and contaminated spectra when combining inputs into a single spectrum (in combine_1d). This PR also adds an option to perform a median filter on the 1D master background spectrum produced in both master_background and master_background MOS (by default this is off).

Tasks

• [x] request a review from someone specific, to avoid making the maintainers review every PR
• [x] add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
• [x] Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • [x] write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • [x] update or add relevant tests
  • [x] update relevant docstrings and / or docs/ page
  • [x] start a regression test and include a link to the running job (click here for instructions)
  • [x] Do truth files need to be updated ("okified")?
    • [x] after the reviewer has approved these changes, run okify_regtests to update the truth files
• [x] if a JIRA ticket exists, make sure it is resolved properly

news fragment change types...

- ``changes/.general.rst``: infrastructure or miscellaneous change - ``changes/.docs.rst`` - ``changes/.stpipe.rst`` - ``changes/.datamodels.rst`` - ``changes/.scripts.rst`` - ``changes/.fits_generator.rst`` - ``changes/.set_telescope_pointing.rst`` - ``changes/.pipeline.rst`` ## stage 1 - ``changes/.group_scale.rst`` - ``changes/.dq_init.rst`` - ``changes/.emicorr.rst`` - ``changes/.saturation.rst`` - ``changes/.ipc.rst`` - ``changes/.firstframe.rst`` - ``changes/.lastframe.rst`` - ``changes/.reset.rst`` - ``changes/.superbias.rst`` - ``changes/.refpix.rst`` - ``changes/.linearity.rst`` - ``changes/.rscd.rst`` - ``changes/.persistence.rst`` - ``changes/.dark_current.rst`` - ``changes/.charge_migration.rst`` - ``changes/.jump.rst`` - ``changes/.clean_flicker_noise.rst`` - ``changes/.ramp_fitting.rst`` - ``changes/.gain_scale.rst`` ## stage 2 - ``changes/.assign_wcs.rst`` - ``changes/.badpix_selfcal.rst`` - ``changes/.msaflagopen.rst`` - ``changes/.nsclean.rst`` - ``changes/.imprint.rst`` - ``changes/.background.rst`` - ``changes/.extract_2d.rst`` - ``changes/.master_background.rst`` - ``changes/.wavecorr.rst`` - ``changes/.srctype.rst`` - ``changes/.straylight.rst`` - ``changes/.wfss_contam.rst`` - ``changes/.flatfield.rst`` - ``changes/.fringe.rst`` - ``changes/.pathloss.rst`` - ``changes/.barshadow.rst`` - ``changes/.photom.rst`` - ``changes/.pixel_replace.rst`` - ``changes/.resample_spec.rst`` - ``changes/.residual_fringe.rst`` - ``changes/.cube_build.rst`` - ``changes/.extract_1d.rst`` - ``changes/.resample.rst`` ## stage 3 - ``changes/.assign_mtwcs.rst`` - ``changes/.mrs_imatch.rst`` - ``changes/.tweakreg.rst`` - ``changes/.skymatch.rst`` - ``changes/.exp_to_source.rst`` - ``changes/.outlier_detection.rst`` - ``changes/.tso_photometry.rst`` - ``changes/.stack_refs.rst`` - ``changes/.align_refs.rst`` - ``changes/.klip.rst`` - ``changes/.spectral_leak.rst`` - ``changes/.source_catalog.rst`` - ``changes/.combine_1d.rst`` - ``changes/.ami.rst`` ## other - ``changes/.wfs_combine.rst`` - ``changes/.white_light.rst`` - ``changes/.cube_skymatch.rst`` - ``changes/.engdb_tools.rst`` - ``changes/.guider_cds.rst``

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 93.97590% with 5 lines in your changes missing coverage. Please review.

Project coverage is 64.52%. Comparing base (4d8e589) to head (72cb6f0). Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...st/master_background/master_background_mos_step.py	44.44%	5 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #8932 +/- ## ========================================== - Coverage 64.52% 64.52% -0.01% ========================================== Files 375 375 Lines 38738 38739 +1 ========================================== Hits 24997 24997 - Misses 13741 13742 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[melanieclarke]

Everything looks good to me now, other than the potential conflict with #8847. I will update from main and run the full set of regression tests.

Tests are running here: https://github.com/spacetelescope/RegressionTests/actions/runs/11902214724