search

spacetelescope / stsynphot_refactor

Synthetic photometry using Astropy for HST and JWST
http://stsynphot.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
13 stars 10 forks source link

Test against new TMG #146

Closed sean-lockwood closed 3 years ago

sean-lockwood commented 3 years ago

Description

Please run the test suite using the latest TMG, 4cm1612bm_tmg.fits, delivered to CRDS on 2020-12-22. This file contains a new index columns ("col0"), and we should make sure the inclusion of this (presumably) dummy column doesn't break stsynphot in unexpected ways.

The corresponding TMC is 55104091m_tmc.fits.

pllim commented 3 years ago

Results from #147 

============================= test session starts ==============================
platform linux -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
cachedir: .tox/py39-test-alldeps-cov/.pytest_cache

Running tests with stsynphot version 1.0.1.dev72+gf78d89c.
Running tests in stsynphot docs.

Date: 2021-05-18T19:35:43

Platform: Linux-5.4.0-1047-azure-x86_64-with-glibc2.31

Executable: /home/runner/work/stsynphot_refactor/stsynphot_refactor/.tox/py39-test-alldeps-cov/bin/python

Full Python Version: 
3.9.5 (default, May  4 2021, 07:05:36) 
[GCC 9.3.0]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions: 
Numpy: 1.20.3
Scipy: 1.6.3
Matplotlib: 3.4.2
beautifulsoup4: 4.9.3
astropy: 4.2.1
synphot: 1.0.1

Using Astropy options: remote_data: any.

rootdir: /home/runner/work/stsynphot_refactor/stsynphot_refactor, configfile: setup.cfg
plugins: hypothesis-6.13.0, astropy-header-0.1.2, filter-subpackage-0.1.1, cov-2.12.0, arraydiff-0.3, remotedata-0.3.2, doctestplus-0.9.0, openfiles-0.5.0, ci-watson-0.5
collected 185 items

../../stsynphot/tests/test_catalog.py ..........                         [  5%]
../../stsynphot/tests/test_config.py .x.....                             [  9%]
../../stsynphot/tests/test_observationmode.py ............FF.F....       [ 20%]
../../stsynphot/tests/test_parser.py ................................... [ 38%]
.................................                                        [ 56%]
../../stsynphot/tests/test_spectrum.py ............FFFFFFFF............. [ 74%]
..                                                                       [ 75%]
../../stsynphot/tests/test_stio.py .............                         [ 82%]
../../stsynphot/tests/test_tables.py ...........                         [ 88%]
../../stsynphot/tests/test_wavetable.py .........                        [ 93%]
../../docs/index.rst .                                                   [ 94%]
../../docs/stsynphot/appendixa.rst .                                     [ 94%]
../../docs/stsynphot/appendixb_inflight.rst .                            [ 95%]
../../docs/stsynphot/appendixb_legacy.rst .                              [ 95%]
../../docs/stsynphot/appendixb_nonhst.rst .                              [ 96%]
../../docs/stsynphot/appendixb_specialkey.rst .                          [ 96%]
../../docs/stsynphot/appendixc.rst .                                     [ 97%]
../../docs/stsynphot/from_pysyn_iraf.rst .                               [ 97%]
../../docs/stsynphot/obsmode.rst .                                       [ 98%]
../../docs/stsynphot/refdata.rst .                                       [ 98%]
../../docs/stsynphot/spectrum.rst .                                      [ 99%]
../../docs/stsynphot/tutorials.rst .                                     [100%]

=================================== FAILURES ===================================
_____________________ TestObservationMode.test_throughput ______________________

self = <stsynphot.tests.test_observationmode.TestObservationMode object at 0x7f3d99315670>

    def test_throughput(self):
        t = self.obsmode.throughput
        w = t.waveset[::1000]
        np.testing.assert_allclose(
            w.value,
            [500, 3797, 4797, 5797, 6797, 7797, 8797, 9797, 10796])
>       np.testing.assert_allclose(
            t(w).value,
            [0, 6.87477055e-06, 2.14807644e-01, 2.85725420e-01, 7.34488102e-07,
             2.96078061e-08, 3.57507695e-08, 9.99538418e-07, 0], rtol=1e-3)
E       AssertionError: 
E       Not equal to tolerance rtol=0.001, atol=0
E       
E       Mismatched elements: 7 / 9 (77.8%)
E       Max absolute difference: 0.00142971
E       Max relative difference: 0.01703023
E        x: array([0.000000e+00, 6.757692e-06, 2.133779e-01, 2.845666e-01,
E              7.306088e-07, 2.941201e-08, 3.548093e-08, 9.848318e-07,
E              0.000000e+00])
E        y: array([0.000000e+00, 6.874771e-06, 2.148076e-01, 2.857254e-01,
E              7.344881e-07, 2.960781e-08, 3.575077e-08, 9.995384e-07,
E              0.000000e+00])

../../stsynphot/tests/test_observationmode.py:161: AssertionError
_____________________ TestObservationMode.test_sensitivity _____________________

self = <stsynphot.tests.test_observationmode.TestObservationMode object at 0x7f3d992e07f0>

    def test_sensitivity(self):
        sens = self.obsmode.sensitivity
        w = sens.waveset[::1000]
        np.testing.assert_allclose(
            w.value,
            [500, 3797, 4797, 5797, 6797, 7797, 8797, 9797, 10796])
>       np.testing.assert_allclose(
            sens(w).value,
            [0, 5.94476276e+10, 2.34668703e+15, 3.77214087e+15, 1.13694055e+10,
             5.25738140e+08, 7.16235278e+08, 2.23012046e+10, 0], rtol=1e-3)
E       AssertionError: 
E       Not equal to tolerance rtol=0.001, atol=0
E       
E       Mismatched elements: 7 / 9 (77.8%)
E       Max absolute difference: 1.56195336e+13
E       Max relative difference: 0.01703044
E        x: array([0.000000e+00, 5.843521e+10, 2.331067e+15, 3.756842e+15,
E              1.130935e+10, 5.222613e+08, 7.108290e+08, 2.197307e+10,
E              0.000000e+00])
E        y: array([0.000000e+00, 5.944763e+10, 2.346687e+15, 3.772141e+15,
E              1.136941e+10, 5.257381e+08, 7.162353e+08, 2.230120e+10,
E              0.000000e+00])

../../stsynphot/tests/test_observationmode.py:172: AssertionError
____________________ TestObservationMode.test_thermal_spec _____________________

self = <stsynphot.tests.test_observationmode.TestObservationMode object at 0x7f3d9942d460>

    def test_thermal_spec(self):
        """Also see TestThermalObservationMode.
        Whitespace in obsmode should not matter.
        Flux values are from ASTROLIB PYSYNPHOT, except the first value,
        which is supposed to be zero.

        """
        obsmode = observationmode.ObservationMode(
            'wfc3, ir, f153m', graphtable=GT_FILE, comptable=CP_FILE)
        thsp = obsmode.thermal_spectrum(thermtable=TH_FILE)
        w = [6898, 7192, 7486, 7780, 8630, 11190, 13790, 15670, 17954.90234375]
>       np.testing.assert_allclose(
            thsp(w).value,
            [2.911303e-30, 9.52169734e-29, 2.32066138e-27, 4.38458258e-26,
             1.86992105e-21, 2.90270169e-15, 3.42776351e-11, 2.15179718e-08,
             7.24300701e-09], rtol=5e-3)
E       AssertionError: 
E       Not equal to tolerance rtol=0.005, atol=0
E       
E       Mismatched elements: 1 / 9 (11.1%)
E       Max absolute difference: 1.51369191e-10
E       Max relative difference: 0.00703455
E        x: array([2.911318e-30, 9.554516e-29, 2.328319e-27, 4.398529e-26,
E              1.863447e-21, 2.897122e-15, 3.428160e-11, 2.136660e-08,
E              7.252959e-09])
E        y: array([2.911303e-30, 9.521697e-29, 2.320661e-27, 4.384583e-26,
E              1.869921e-21, 2.902702e-15, 3.427764e-11, 2.151797e-08,
E              7.243007e-09])

../../stsynphot/tests/test_observationmode.py:196: AssertionError
_ TestObservationSpectralElement.test_uresp[acs,hrc,f555w-2.9788972414188295e-19] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98549790>
obsmode = 'acs,hrc,f555w', ans = 2.9788972414188295e-19

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 6.20605744e-21
E       Max relative difference: 0.02083341
E        x: array(3.040958e-19)
E        y: array(2.978897e-19)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[acs,sbc,f125lp-1.7218083497870695e-17] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d994c9100>
obsmode = 'acs,sbc,f125lp', ans = 1.7218083497870695e-17

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 4.41973305e-18
E       Max relative difference: 0.25669135
E        x: array(1.279835e-17)
E        y: array(1.721808e-17)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[acs,wfc1,f555w,f814w-1.7485647715025005e-13] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98eadeb0>
obsmode = 'acs,wfc1,f555w,f814w', ans = 1.7485647715025005e-13

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 5.56903044e-15
E       Max relative difference: 0.03184915
E        x: array(1.804255e-13)
E        y: array(1.748565e-13)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[cos,boa,fuv,g130m,c1309-3.810821982440199e-15] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98eafdf0>
obsmode = 'cos,boa,fuv,g130m,c1309', ans = 3.810821982440199e-15

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 5.72227258e-16
E       Max relative difference: 0.15015849
E        x: array(4.383049e-15)
E        y: array(3.810822e-15)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[stis,ccd,f25ndq1,a2d4,mjd#55555-3.0597787848106823e-18] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98154610>
obsmode = 'stis,ccd,f25ndq1,a2d4,mjd#55555', ans = 3.0597787848106823e-18

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 5.15629578e-21
E       Max relative difference: 0.00168519
E        x: array(3.064935e-18)
E        y: array(3.059779e-18)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[wfc3,ir,f140w-1.4737148727216957e-20] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98139940>
obsmode = 'wfc3,ir,f140w', ans = 1.4737148727216957e-20

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 1.15103518e-23
E       Max relative difference: 0.00078104
E        x: array(1.474866e-20)
E        y: array(1.473715e-20)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[wfc3,uvis1,f395n-5.9433862614148255e-18] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98d0faf0>
obsmode = 'wfc3,uvis1,f395n', ans = 5.9433862614148255e-18

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 1.5556555e-20
E       Max relative difference: 0.00261746
E        x: array(5.958943e-18)
E        y: array(5.943386e-18)

../../stsynphot/tests/test_spectrum.py:212: AssertionError
_ TestObservationSpectralElement.test_uresp[wfc3,uvis2,fq924n-6.163270991174246e-18] _

self = <stsynphot.tests.test_spectrum.TestObservationSpectralElement object at 0x7f3d98bc3640>
obsmode = 'wfc3,uvis2,fq924n', ans = 6.163270991174246e-18

    @pytest.mark.parametrize(
        ('obsmode', 'ans'),
        [('acs,hrc,f555w', 2.9788972414188295e-19),
         ('acs,sbc,f125lp', 1.7218083497870695e-17),
         ('acs,wfc1,f555w,f814w', 1.7485647715025005e-13),
         ('cos,boa,fuv,g130m,c1309', 3.8108219824401987e-15),
         ('stis,ccd,f25ndq1,a2d4,mjd#55555', 3.0597787848106823e-18),
         ('wfc3,ir,f140w', 1.4737148727216957e-20),
         ('wfc3,uvis1,f395n', 5.9433862614148255e-18),
         ('wfc3,uvis2,fq924n', 6.1632709911742462e-18),
         ('wfpc2,1,a2d7,f300w,cont#49892.0', 6.3011E-17),
         ('wfpc2,f555w', 4.8967453103320938e-19)])
    def test_uresp(self, obsmode, ans):
        """Unit response for different detector settings."""
        with warnings.catch_warnings():
            warnings.filterwarnings(
                'ignore', message=r'.*not valid flux unit.*',
                category=AstropyUserWarning)
            obs = spectrum.band(
                obsmode, graphtable=GT_FILE, comptable=CP_FILE)

>       np.testing.assert_allclose(
            obs.unit_response(obs.area).value, ans, rtol=1e-4)
E       AssertionError: 
E       Not equal to tolerance rtol=0.0001, atol=0
E       
E       Mismatched elements: 1 / 1 (100%)
E       Max absolute difference: 1.96818261e-19
E       Max relative difference: 0.03193406
E        x: array(6.360089e-18)
E        y: array(6.163271e-18)

../../stsynphot/tests/test_spectrum.py:212: AssertionError

============= 11 failed, 173 passed, 1 xfailed in 89.86s (0:01:29) =============
pllim commented 3 years ago

Resolved: https://github.com/spacetelescope/stsynphot_refactor/pull/147#issuecomment-843518155