spacetelescope / pysiaf

Handling of Science Instrument Aperture Files (SIAF) for space telescopes
https://pysiaf.readthedocs.io
BSD 3-Clause "New" or "Revised" License
13 stars 28 forks source link

Update test_nirspec.py #255

Open Witchblade101 opened 2 years ago

Witchblade101 commented 2 years ago

After importing a PRD with the recent NRS SIAF changes, test_nirspec.py fails two transform tests:

======================================================= short test summary info ======================================================== FAILED test_nirspec.py::test_nirspec_aperture_transforms - assert 0.27277224621730056 < 0.2 FAILED test_nirspec.py::test_nirspec_slit_transformations - assert 0.27277224621730056 < 0.1 ==================================================== 2 failed, 30 passed in 25.56s =====================================================

@charlesrp provided the following explanation:

The real problem is with the test function. Because of the micro-shutter array, in imaging mode only about 1/2 of each of the NIRSpec detectors can actually see the sky. So the transforms are really only constrained and used on those parts of the detectors. Unfortunately the test function was set up to use points over the whole regions of the detectors including parts that never see the sky in imaging mode. Pre-launch, the discrepancy in the extrapolated region wasn’t too bad, but since the on-sky solution has more structure, the extrapolation goes bad more quickly.

The best solution would be to fix the test function so it only tests points in the valid region.

Tracy has code that includes the needed fix and also adds some tests for other transforms that were not previously tested.

So we should discuss how to get this improved test code merged into PySiaf. If that can’t be done in the short term, a waiver might be appropriate.

So we have two options on how to proceed and be able to do valid releases of pysiaf:

  1. Increase pixel_threshold temporarily to allow the tests to pass, similarly as to what has been done for FGS as part of #186
  2. If @tracy-beck has the new code ready to go we add it to this release.

1 is obviously not great, but will allow us to get the new release out for NIRCam and MIRI. 2 has a lot more risks, in that Tracy is probably extremely busy, may need more time to test her changes, and will probably want some help creating the PR.

Even if we do decide to do option 1, we need option 2 before the end of commissioning.

Witchblade101 commented 2 years ago

I created a branch to test option 1 above on my personal fork. The two tests that were failing now pass, but two different ones now fail:

======================================================= short test summary info ======================================================== FAILED test_miri.py::test_against_test_data - AssertionError: FAILED test_nirspec.py::test_against_test_data - AssertionError: Failed for NRS1_FULL_OSS ==================================================== 2 failed, 30 passed in 29.73s =====================================================

So it's not quite the quick fix we were hoping for, as far as getting a new release out.

Witchblade101 commented 2 years ago

Tried the quick and dirty fix again and it works. I may just have been more careful in other steps this time:

========================================================= test session starts ========================================================== platform darwin -- Python 3.9.12, pytest-7.1.1, pluggy-1.0.0 -- /Users/dlong/miniconda3/envs/pysiaf/bin/python cachedir: .pytest_cache rootdir: /Users/dlong/pysiaf, configfile: setup.cfg collected 32 items

test_aperture.py::test_idl_to_tel PASSED [ 3%] test_aperture.py::test_hst_fgs_idl_to_tel PASSED [ 6%] test_aperture.py::test_jwst_aperture_transforms PASSED [ 9%] test_aperture.py::test_jwst_aperture_vertices PASSED [ 12%] test_aperture.py::test_raw_transformations PASSED [ 15%] test_aperture.py::test_jwst_sky_transformations PASSED [ 18%] test_hst.py::test_hst_aperture_init PASSED [ 21%] test_hst.py::test_hst_siaf PASSED [ 25%] test_hst.py::test_hst_amudotrep PASSED [ 28%] test_iando.py::test_write_jwst_siaf_xml PASSED [ 31%] test_iando.py::test_write_jwst_siaf_xlsx PASSED [ 34%] test_match_v2v3.py::test_match_v2v3 PASSED [ 37%] test_miri.py::test_against_test_data PASSED [ 40%] test_nirspec.py::test_against_test_data PASSED [ 43%] test_nirspec.py::test_nirspec_aperture_transforms PASSED [ 46%] test_nirspec.py::test_nirspec_slit_transformations PASSED [ 50%] test_nirspec.py::test_sci_det_consistency PASSED [ 53%] test_plotting.py::test_aperture_plotting PASSED [ 56%] test_polynomial.py::test_poly PASSED [ 59%] test_polynomial.py::test_RotateCoeffs PASSED [ 62%] test_polynomial.py::test_two_step PASSED [ 65%] test_polynomial.py::test_invert PASSED [ 68%] test_polynomial.py::test_ShiftCoeffs PASSED [ 71%] test_projection.py::test_tangent_plane_projection_roundtrip PASSED [ 75%] test_projection.py::test_project_to_tangent_plane PASSED [ 78%] test_projection.py::test_deproject_from_tangent_plane PASSED [ 81%] test_rotations.py::test_attitude PASSED [ 84%] test_rotations.py::test_attitude_matrix PASSED [ 87%] test_rotations.py::test_sky_to_tel PASSED [ 90%] test_rotations.py::test_axial_rotation PASSED [ 93%] test_rotations.py::test_unit_vector_from_cartesian PASSED [ 96%] test_tools.py::test_jwst_fgs_to_fgs_matrix PASSED [100%]

========================================================= 32 passed in 30.31s ==========================================================