search

desihub / redrock

Redshift fitting for spectroperfectionism
BSD 3-Clause "New" or "Revised" License
21 stars 13 forks source link

add TEMFILnn keywords with full template filenames #295

Closed sbailey closed 2 months ago

sbailey commented 2 months ago

@moustakas this is the followup PR to #290 to save the template filenames in the output headers as TEMFILnn keywods to support non-standard template names. This is supported by redrock.templates.load_templates_from_header to read them back in.

Example:

echo $RR_TEMPLATE_DIR
cat $RR_TEMPLATE_DIR/templates-test.txt
rrdesi -n 16 \
  -i $CFS/desi/spectro/redux/iron/tiles/cumulative/80605/20210205/coadd-6-80605-thru20210205.fits \
  -t $RR_TEMPLATE_DIR/templates-test.txt \
  -o $SCRATCH/redrock/redrock-test-keywords.fits
fitsheader -e 0 $SCRATCH/redrock/redrock-test-keywords.fits

Results:

/global/common/software/desi/users/sjbailey/redrock-templates
rrtemplate-GALAXY-None-v2.6.fits
rrtemplate-QSO-SuperDuperWow.fits
...
TEMNAM00= 'GALAXY  '                                                            
TEMVER00= '2.6     '                                                            
TEMFIL00= 'rrtemplate-GALAXY-None-v2.6.fits'                                    
TEMNAM01= 'QSO:::LOZ'                                                           
TEMVER01= '1.0     '                                                            
TEMFIL01= 'rrtemplate-QSO-SuperDuperWow.fits'                                   
...
DEPNAM19= 'RR_TEMPLATE_DIR'                                                     
DEPVER19= '/global/common/software/desi/users/sjbailey/redrock-templates'

Loading those templates using the header:

import fitsio
from redrock.templates import load_templates_from_header
hdr = fitsio.read_header('/pscratch/sd/s/sjbailey/redrock/redrock-test-keywords.fits')
templates = load_templates_from_header(hdr)

Results:

Reading templates from ['/global/common/software/desi/users/sjbailey/redrock-templates/rrtemplate-GALAXY-None-v2.6.fits', '/global/common/software/desi/users/sjbailey/redrock-templates/rrtemplate-QSO-SuperDuperWow.fits']
INFO: rrtemplate-GALAXY-None-v2.6.fits GALAXY  PCA IGM=Calura12 z=-0.005-1.6997
INFO: rrtemplate-QSO-SuperDuperWow.fits QSO LOZ PCA IGM=Calura12 z=0.05-1.5983

Note: this does not implement the other idea in PR #290 to support past runs without TEMFILnn keywords, where it reads every template it can find and tries to match it up to the TEMNAMnn and TEMVERnn keywords. Let's save that for a different PR if we really really need it.

coveralls commented 2 months ago

Coverage Status

coverage: 39.17% (+0.2%) from 39.015% when pulling 8e6b8558f29ccaaaf7e1edf9b2695ec407efa8df on template_keywords into e573662c3eaddb30761de2e3ad29d8af7b59883a on main.

moustakas commented 2 months ago

Thanks for this @sbailey. I tested with your code example but then pushed a little further by looking into the details file. I ran:

rrdesi -n 16   -i $CFS/desi/spectro/redux/iron/tiles/cumulative/80605/20210205/coadd-6-80605-thru20210205.fits   \
  -t /global/common/software/desi/users/sjbailey/redrock-templates/templates-test.txt   -o ./redrock-test-keywords.fits \
  -d ./rrdetails-test-keywords.h5

(Note that I still get the warning reported in https://github.com/desihub/redrock/issues/285; do we care?)

Then I did:

import os
import fitsio
from redrock.results import read_zscan

os.environ['RR_TEMPLATE_DIR'] = '/global/common/software/desi/users/sjbailey/redrock-templates'

from redrock.templates import load_templates_from_header
hdr = fitsio.read_header('./redrock-test-keywords.fits')
templates = load_templates_from_header(hdr)

targetid = 39627700813439235
zb, zf = read_zscan('rrdetails-test-keywords.h5', select_targetids=targetid)

The zf catalog looks good (I was a little worried about the None suffix in the filename and curious how it would be rendered in these tables:

zf
<Table length=6>
     targetid             z                   zerr          zwarn        chi2                           zz                    ... npixels spectype subtype ncoeff  znum     deltachi2
      int64            float64              float64         int64      float64                     float64[15]                ...  int64    str6     str3  int64  int64      float64
----------------- ------------------ ---------------------- ----- ------------------ ---------------------------------------- ... ------- -------- ------- ------ ----- ------------------
39627700813439235  0.781713024341458  9.055410159291171e-05     0   11400.4652864784     0.781242631268682 .. 0.7837052096037 ...    7917   GALAXY             10     0  9.875630427151918
39627700813439235 0.7565464557191716 0.00014658988048198835     4 11410.340916905552 0.7555899286377181 .. 0.7580170419616115 ...    7917   GALAXY             10     1 1.4062871262431145
39627700813439235 0.7745208293277615 0.00017853383229488962     4 11411.747204031795 0.7738752764667702 .. 0.7763276693907706 ...    7917   GALAXY             10     2 121.93956746906042
39627700813439235 0.5605627251307392 0.00031652092407527126     0 11533.686771500856 0.5584371740678022 .. 0.5620297427834777 ...    7917      QSO     LOZ      4     3  36.99288884969428
39627700813439235 0.5530205241242334  0.0003055143840742204     4  11570.67966035055  0.5512768056690951 .. 0.554852868030886 ...    7917      QSO     LOZ      4     4 2.6982464944012463
39627700813439235 0.5453194969507216  0.0003680103369916673     4 11573.377906844951 0.5423725915260609 .. 0.5459281275543555 ...    7917      QSO     LOZ      4     5                0.0

And the GALAXY redshift scans look OK, too:

zb[targetid]['GALAXY']
{'redshifts': array([-0.005     , -0.00431244, -0.00362441, ...,  1.69601982,
         1.69788281,  1.69974708]),
 'zchi2': array([11555.48098543, 11555.36519906, 11539.37803745, ...,
        11540.91722658, 11534.47787985, 11522.70192851]),
 'penalty': array([  0.        ,   0.        ,   0.        , ..., 813.07389685,
        903.81428712, 698.60962426]),
 'zcoeff': array([[ 1.34274124e+02, -5.71008174e+01,  3.13608623e+01, ...,
         -6.82913678e-02,  1.50108128e-01, -9.15245239e-01],
        [ 1.12546076e+02, -2.77736499e+01,  1.61918430e+01, ...,
          7.69901407e-01, -6.95947625e-01, -4.04991687e+00],
        [ 1.15502639e+02, -3.18467862e+01,  1.82021868e+01, ...,
          1.59941555e+00, -6.38585697e-01, -3.55679485e+00],
        ...,
        [-2.25821026e+02,  1.80862899e+02,  1.76240165e+03, ...,
          6.00637524e+02, -1.51983534e+02, -1.23513963e+01],
        [-2.42816953e+02,  1.96563101e+02,  1.87536418e+03, ...,
          6.65209584e+02, -1.47798288e+02, -1.34003031e+01],
        [-2.61971649e+02,  2.03541542e+02,  1.99361533e+03, ...,
          5.98000964e+02, -1.55614630e+02, -1.40422525e+01]]),
 'zfit': <Table length=3>
         z                   zerr          zwarn        chi2                           zz                    ...                  coeff                   npixels spectype subtype ncoeff
      float64              float64         int64      float64                     float64[15]                ...               float64[10]                 int64    str6     str3  int64
 ------------------ ---------------------- ----- ------------------ ---------------------------------------- ... ---------------------------------------- ------- -------- ------- ------
  0.781713024341458  9.055410159291171e-05     0   11400.4652864784     0.781242631268682 .. 0.7837052096037 ...  57.77194056946836 .. -4.527364015044522    7917   GALAXY             10
 0.7565464557191716 0.00014658988048198835     4 11410.340916905552 0.7555899286377181 .. 0.7580170419616115 ... 56.560849112657564 .. -4.096772717390954    7917   GALAXY             10
 0.7745208293277615 0.00017853383229488962     4 11411.747204031795 0.7738752764667702 .. 0.7763276693907706 ... 55.503773624845834 .. -4.767074811200979    7917   GALAXY             10}

But I'm confused why the zfit table for the QSO::LOZ template is empty. Maybe this is expected, but shouldn't it have the info on those fits?

zb[39627700813439235]['QSO:::LOZ']
{'redshifts': array([0.05      , 0.05120955, 0.0524205 , 0.05363284, 0.05484658,
        0.05606172, 0.05727825, 0.05849619, 0.05971553, 0.06093628,
        0.06215843, 0.06338199, 0.06460696, 0.06583333, 0.06706113,
        0.06829033, 0.06952096, 0.070753  , 0.07198646, 0.07322134,
        0.07445764, 0.07569537, 0.07693452, 0.0781751 , 0.07941711,
        0.08066055, 0.08190543, 0.08315173, 0.08439948, 0.08564866,
        0.08689928, 0.08815133, 0.08940484, 0.09065978, 0.09191617,
        0.09317401, 0.0944333 , 0.09569404, 0.09695623, 0.09821987,
        0.09948498, 0.10075153, 0.10201955, 0.10328903, 0.10455997,
        0.10583237, 0.10710624, 0.10838158, 0.10965838, 0.11093666,
        0.11221641, 0.11349764, 0.11478034, 0.11606451, 0.11735017,
        0.11863731, 0.11992593, 0.12121603, 0.12250762, 0.1238007 ,
        0.12509527, 0.12639133, 0.12768888, 0.12898793, 0.13028847,
        0.13159052, 0.13289406, 0.1341991 , 0.13550565, 0.1368137 ,
        0.13812326, 0.13943433, 0.1407469 , 0.14206099, 0.1433766 ,
        0.14469372, 0.14601235, 0.14733251, 0.14865418, 0.14997738,
        0.15130211, 0.15262835, 0.15395613, 0.15528544, 0.15661628,
        0.15794865, 0.15928255, 0.16061799, 0.16195497, 0.16329349,
        0.16463356, 0.16597516, 0.16731831, 0.16866301, 0.17000926,
        0.17135706, 0.17270641, 0.17405732, 0.17540978, 0.1767638 ,
        0.17811938, 0.17947652, 0.18083522, 0.18219549, 0.18355733,
        0.18492073, 0.18628571, 0.18765226, 0.18902038, 0.19039008,
        0.19176136, 0.19313421, 0.19450865, 0.19588467, 0.19726228,
        0.19864147, 0.20002225, 0.20140462, 0.20278859, 0.20417415,
        0.2055613 , 0.20695006, 0.20834041, 0.20973236, 0.21112592,
        0.21252108, 0.21391785, 0.21531623, 0.21671622, 0.21811783,
        0.21952104, 0.22092588, 0.22233233, 0.2237404 , 0.2251501 ,
        0.22656142, 0.22797436, 0.22938893, 0.23080513, 0.23222297,
        0.23364243, 0.23506353, 0.23648627, 0.23791065, 0.23933667,
        0.24076433, 0.24219363, 0.24362459, 0.24505719, 0.24649144,
        0.24792734, 0.2493649 , 0.25080411, 0.25224498, 0.25368751,
        0.2551317 , 0.25657756, 0.25802508, 0.25947427, 0.26092513,
        0.26237766, 0.26383186, 0.26528774, 0.26674529, 0.26820453,
        0.26966544, 0.27112804, 0.27259232, 0.27405829, 0.27552595,
        0.2769953 , 0.27846634, 0.27993908, 0.28141351, 0.28288964,
        0.28436748, 0.28584701, 0.28732825, 0.28881119, 0.29029585,
        0.29178221, 0.29327029, 0.29476007, 0.29625158, 0.29774481,
        0.29923975, 0.30073642, 0.30223481, 0.30373492, 0.30523677,
        0.30674034, 0.30824565, 0.30975269, 0.31126147, 0.31277198,
        0.31428424, 0.31579823, 0.31731397, 0.31883146, 0.3203507 ,
        0.32187168, 0.32339442, 0.32491891, 0.32644516, 0.32797316,
        0.32950293, 0.33103446, 0.33256775, 0.33410281, 0.33563964,
        0.33717824, 0.3387186 , 0.34026075, 0.34180467, 0.34335037,
        0.34489785, 0.34644711, 0.34799816, 0.34955099, 0.35110562,
        0.35266203, 0.35422024, 0.35578024, 0.35734204, 0.35890563,
        0.36047103, 0.36203823, 0.36360724, 0.36517806, 0.36675068,
        0.36832512, 0.36990137, 0.37147943, 0.37305932, 0.37464102,
        0.37622454, 0.37780989, 0.37939707, 0.38098607, 0.38257691,
        0.38416958, 0.38576408, 0.38736042, 0.38895859, 0.39055861,
        0.39216047, 0.39376418, 0.39536973, 0.39697714, 0.39858639,
        0.4001975 , 0.40181047, 0.40342529, 0.40504198, 0.40666052,
        0.40828093, 0.40990321, 0.41152736, 0.41315337, 0.41478126,
        0.41641103, 0.41804267, 0.41967619, 0.4213116 , 0.42294888,
        0.42458806, 0.42622912, 0.42787207, 0.42951692, 0.43116366,
        0.43281229, 0.43446283, 0.43611527, 0.43776961, 0.43942585,
        0.44108401, 0.44274407, 0.44440605, 0.44606994, 0.44773575,
        0.44940348, 0.45107313, 0.4527447 , 0.4544182 , 0.45609362,
        0.45777098, 0.45945026, 0.46113148, 0.46281464, 0.46449974,
        0.46618678, 0.46787576, 0.46956669, 0.47125956, 0.47295439,
        0.47465117, 0.4763499 , 0.47805059, 0.47975324, 0.48145785,
        0.48316442, 0.48487296, 0.48658347, 0.48829595, 0.4900104 ,
        0.49172682, 0.49344523, 0.49516561, 0.49688797, 0.49861232,
        0.50033866, 0.50206698, 0.50379729, 0.5055296 , 0.50726391,
        0.50900021, 0.51073851, 0.51247881, 0.51422112, 0.51596543,
        0.51771176, 0.5194601 , 0.52121045, 0.52296281, 0.5247172 ,
        0.5264736 , 0.52823203, 0.52999249, 0.53175497, 0.53351949,
        0.53528603, 0.53705461, 0.53882523, 0.54059789, 0.54237259,
        0.54414934, 0.54592813, 0.54770897, 0.54949186, 0.55127681,
        0.55306381, 0.55485287, 0.55664399, 0.55843717, 0.56023242,
        0.56202974, 0.56382913, 0.56563059, 0.56743413, 0.56923974,
        0.57104744, 0.57285722, 0.57466908, 0.57648303, 0.57829906,
        0.58011719, 0.58193742, 0.58375974, 0.58558416, 0.58741068,
        0.58923931, 0.59107004, 0.59290289, 0.59473784, 0.59657491,
        0.59841409, 0.60025539, 0.60209881, 0.60394436, 0.60579203,
        0.60764183, 0.60949377, 0.61134783, 0.61320403, 0.61506237,
        0.61692285, 0.61878548, 0.62065024, 0.62251716, 0.62438623,
        0.62625745, 0.62813083, 0.63000636, 0.63188406, 0.63376391,
        0.63564594, 0.63753013, 0.63941649, 0.64130502, 0.64319574,
        0.64508862, 0.64698369, 0.64888094, 0.65078038, 0.65268201,
        0.65458582, 0.65649183, 0.65840004, 0.66031044, 0.66222305,
        0.66413785, 0.66605486, 0.66797409, 0.66989552, 0.67181916,
        0.67374502, 0.6756731 , 0.67760341, 0.67953593, 0.68147068,
        0.68340766, 0.68534687, 0.68728832, 0.689232  , 0.69117792,
        0.69312608, 0.69507648, 0.69702914, 0.69898404, 0.70094119,
        0.7029006 , 0.70486227, 0.70682619, 0.70879238, 0.71076083,
        0.71273155, 0.71470455, 0.71667981, 0.71865735, 0.72063716,
        0.72261926, 0.72460364, 0.72659031, 0.72857926, 0.73057051,
        0.73256405, 0.73455989, 0.73655802, 0.73855846, 0.7405612 ,
        0.74256625, 0.74457361, 0.74658328, 0.74859527, 0.75060957,
        0.7526262 , 0.75464515, 0.75666642, 0.75869002, 0.76071595,
        0.76274422, 0.76477482, 0.76680776, 0.76884305, 0.77088068,
        0.77292065, 0.77496298, 0.77700766, 0.77905469, 0.78110408,
        0.78315584, 0.78520995, 0.78726643, 0.78932529, 0.79138651,
        0.79345011, 0.79551608, 0.79758444, 0.79965517, 0.8017283 ,
        0.80380381, 0.80588171, 0.807962  , 0.8100447 , 0.81212979,
        0.81421728, 0.81630718, 0.81839948, 0.8204942 , 0.82259133,
        0.82469087, 0.82679283, 0.82889722, 0.83100403, 0.83311326,
        0.83522493, 0.83733902, 0.83945556, 0.84157453, 0.84369594,
        0.84581979, 0.8479461 , 0.85007485, 0.85220605, 0.85433971,
        0.85647583, 0.85861441, 0.86075545, 0.86289895, 0.86504493,
        0.86719338, 0.8693443 , 0.87149771, 0.87365359, 0.87581195,
        0.87797281, 0.88013615, 0.88230198, 0.88447031, 0.88664113,
        0.88881446, 0.89099029, 0.89316863, 0.89534947, 0.89753283,
        0.89971871, 0.9019071 , 0.90409801, 0.90629145, 0.90848741,
        0.9106859 , 0.91288693, 0.91509049, 0.91729659, 0.91950523,
        0.92171641, 0.92393014, 0.92614643, 0.92836526, 0.93058665,
        0.9328106 , 0.93503711, 0.93726619, 0.93949783, 0.94173205,
        0.94396884, 0.9462082 , 0.94845015, 0.95069468, 0.95294179,
        0.95519149, 0.95744379, 0.95969868, 0.96195616, 0.96421625,
        0.96647894, 0.96874423, 0.97101214, 0.97328266, 0.97555579,
        0.97783154, 0.98010992, 0.98239092, 0.98467454, 0.9869608 ,
        0.98924969, 0.99154122, 0.99383538, 0.99613219, 0.99843165,
        1.00073375, 1.00303851, 1.00534592, 1.00765599, 1.00996872,
        1.01228412, 1.01460218, 1.01692291, 1.01924632, 1.0215724 ,
        1.02390116, 1.0262326 , 1.02856673, 1.03090355, 1.03324306,
        1.03558527, 1.03793017, 1.04027778, 1.04262809, 1.0449811 ,
        1.04733683, 1.04969527, 1.05205643, 1.05442031, 1.05678691,
        1.05915623, 1.06152829, 1.06390308, 1.0662806 , 1.06866087,
        1.07104387, 1.07342962, 1.07581812, 1.07820937, 1.08060338,
        1.08300014, 1.08539966, 1.08780195, 1.09020701, 1.09261483,
        1.09502543, 1.09743881, 1.09985496, 1.1022739 , 1.10469563,
        1.10712014, 1.10954745, 1.11197756, 1.11441046, 1.11684617,
        1.11928468, 1.121726  , 1.12417014, 1.12661709, 1.12906686,
        1.13151945, 1.13397486, 1.13643311, 1.13889418, 1.14135809,
        1.14382484, 1.14629443, 1.14876687, 1.15124215, 1.15372029,
        1.15620128, 1.15868513, 1.16117184, 1.16366141, 1.16615385,
        1.16864916, 1.17114735, 1.17364842, 1.17615236, 1.17865919,
        1.18116891, 1.18368152, 1.18619703, 1.18871543, 1.19123673,
        1.19376094, 1.19628805, 1.19881808, 1.20135102, 1.20388688,
        1.20642566, 1.20896736, 1.21151199, 1.21405956, 1.21661006,
        1.21916349, 1.22171987, 1.22427919, 1.22684146, 1.22940669,
        1.23197486, 1.234546  , 1.2371201 , 1.23969716, 1.24227719,
        1.24486019, 1.24744617, 1.25003513, 1.25262707, 1.255222  ,
        1.25781991, 1.26042082, 1.26302473, 1.26563163, 1.26824154,
        1.27085445, 1.27347037, 1.27608931, 1.27871126, 1.28133624,
        1.28396424, 1.28659526, 1.28922932, 1.29186641, 1.29450654,
        1.29714971, 1.29979592, 1.30244518, 1.3050975 , 1.30775287,
        1.3104113 , 1.31307279, 1.31573734, 1.31840497, 1.32107567,
        1.32374945, 1.3264263 , 1.32910624, 1.33178927, 1.33447539,
        1.3371646 , 1.33985691, 1.34255232, 1.34525083, 1.34795246,
        1.3506572 , 1.35336505, 1.35607602, 1.35879011, 1.36150734,
        1.36422769, 1.36695117, 1.3696778 , 1.37240756, 1.37514047,
        1.37787652, 1.38061573, 1.38335809, 1.38610362, 1.3888523 ,
        1.39160415, 1.39435918, 1.39711737, 1.39987874, 1.4026433 ,
        1.40541104, 1.40818196, 1.41095608, 1.41373339, 1.41651391,
        1.41929762, 1.42208455, 1.42487468, 1.42766803, 1.4304646 ,
        1.43326438, 1.43606739, 1.43887364, 1.44168311, 1.44449582,
        1.44731177, 1.45013097, 1.45295341, 1.4557791 , 1.45860805,
        1.46144026, 1.46427573, 1.46711446, 1.46995647, 1.47280175,
        1.47565031, 1.47850214, 1.48135727, 1.48421568, 1.48707739,
        1.48994239, 1.49281069, 1.4956823 , 1.49855722, 1.50143544,
        1.50431699, 1.50720185, 1.51009003, 1.51298154, 1.51587639,
        1.51877456, 1.52167608, 1.52458094, 1.52748915, 1.5304007 ,
        1.53331561, 1.53623388, 1.5391555 , 1.5420805 , 1.54500886,
        1.5479406 , 1.55087571, 1.55381421, 1.55675609, 1.55970136,
        1.56265002, 1.56560208, 1.56855754, 1.5715164 , 1.57447868,
        1.57744436, 1.58041346, 1.58338598, 1.58636193, 1.5893413 ,
        1.59232411, 1.59531035, 1.59830003]),
 'zchi2': array([12040.24868543, 12032.88902967, 12088.60562273, 12118.54983675,
        12170.33928401, 12135.5431129 , 12089.16377735, 12070.58575354,
        12083.58819488, 12095.65348958, 12115.66472215, 12133.83867878,
        12185.3575425 , 12209.03385554, 12155.62958714, 12134.40330101,
        12117.80631031, 12140.85474792, 12156.09034218, 12156.61430883,
        12150.74619152, 12171.5851133 , 12173.32130562, 12159.15839925,
        12144.63176745, 12073.55762621, 12092.25167761, 12145.18100661,
        12138.81632808, 12134.95353516, 12108.00738491, 12117.64766938,
        12105.59817527, 12088.44382468, 12128.70889271, 12140.62029092,
        12135.90148545, 12121.61419258, 12167.79343424, 12208.25244276,
        12216.28236454, 12158.96713292, 12103.92273239, 12136.01505326,
        12129.24287302, 12116.59095886, 12163.05167528, 12239.72098725,
        12291.52886933, 12278.06354292, 12234.03894976, 12052.66727292,
        12067.89767673, 12188.96834321, 12206.12613034, 12234.77190999,
        12233.97908588, 12255.59833573, 12335.8242864 , 12288.78994205,
        12290.09948709, 12326.49087988, 12355.75162047, 12284.35675086,
        12266.59492042, 12269.61303432, 12266.22914036, 12288.32495812,
        12249.98690115, 12266.01283512, 12247.16304291, 12237.74556836,
        12199.27478302, 12198.61191097, 12198.94194994, 12220.86696492,
        12203.75432654, 12190.10865289, 12219.22879646, 12258.56946767,
        12273.97046301, 12233.98286555, 12223.57226259, 12178.52686378,
        12161.09796339, 12143.85366659, 12103.09897746, 12087.30327594,
        12159.03842367, 12223.79459023, 12259.90831415, 12265.4561685 ,
        12236.80171503, 12235.15270837, 12287.80387698, 12351.06255222,
        12390.3734697 , 12432.37856343, 12466.70773431, 12458.80732599,
        12392.53764819, 12449.49906133, 12526.1549148 , 12531.7140534 ,
        12502.4609218 , 12529.40344764, 12623.66113542, 12657.10612212,
        12654.40484117, 12625.53852911, 12660.62737911, 12597.13573909,
        12586.28297297, 12661.21980155, 12700.3417668 , 12782.45486157,
        12840.0003378 , 12871.53563169, 12866.07079239, 12835.09708139,
        12815.50477067, 12852.76130062, 12930.83279962, 12913.01484482,
        12947.22110236, 13035.18255948, 13064.74113275, 13054.64533249,
        13017.90400137, 13033.23684179, 13023.01307044, 13048.97277698,
        13090.69078635, 13133.84172286, 13125.26614457, 13121.05593022,
        13197.16597941, 13155.7764247 , 13189.06476105, 13250.86978255,
        13212.30391104, 13215.63283444, 13217.89823223, 13306.2227047 ,
        13394.23324956, 13417.81975806, 13471.68124073, 13409.53025539,
        13401.98198079, 13353.47400684, 13316.95965043, 13341.0993738 ,
        13298.78121289, 13403.21804077, 13502.79384886, 13520.72863485,
        13479.09198446, 13349.68987056, 13296.43478362, 13442.08085358,
        13534.37954221, 13525.92275473, 13574.05950299, 13623.40773422,
        13658.70691273, 13612.09745285, 13655.09573624, 13719.42178417,
        13804.28528681, 13821.59401645, 13907.06738668, 13975.69940324,
        13960.71637152, 13962.91093156, 13961.98368333, 14024.31593409,
        14071.81422429, 14142.28113498, 14219.76074539, 14237.69804881,
        14245.0246375 , 14203.10217418, 14157.69793792, 14227.47864834,
        14107.33868439, 14160.404282  , 14153.50726556, 14145.17082678,
        14219.57494243, 14082.75833756, 14033.17763714, 14084.27708572,
        13973.26297382, 13843.24105325, 13831.67291417, 13895.27945047,
        13908.41215962, 13808.48546875, 13926.80419421, 13981.41072414,
        13945.70138636, 13919.85770437, 13909.41907761, 14026.70008737,
        14010.85496228, 13974.28345536, 14022.51629239, 14050.93024374,
        14033.09882124, 14057.70234022, 14024.99597529, 13945.25087828,
        13960.67031381, 13942.45042028, 13947.28549073, 14015.90276141,
        14054.61406551, 13959.88783647, 13988.62235919, 13976.05790153,
        13983.77144876, 14111.1265264 , 14036.03524777, 14072.00913416,
        14096.78711451, 14049.32926408, 13994.82998971, 14008.64229489,
        13973.42029771, 14069.00427466, 14217.08130244, 14297.0930491 ,
        14147.85938743, 14139.15501028, 14224.84466819, 14204.58475595,
        14267.4884753 , 14331.38169873, 14287.96899048, 14347.41490509,
        14325.26479544, 14305.73515354, 14392.60832863, 14359.99194387,
        14379.50493273, 14388.96189882, 14356.60105989, 14487.07884482,
        14383.27986271, 14288.64639813, 14241.32630193, 14235.07931859,
        14370.20061076, 14416.89744836, 14178.63549492, 14046.74285294,
        14006.42367101, 13875.03900835, 13765.73358553, 13645.35741129,
        13543.91514081, 13526.05391236, 13381.61294074, 13193.2620217 ,
        13115.69798656, 13039.52630667, 12995.3480761 , 12977.65413503,
        13006.39898094, 12952.27160992, 12992.61979221, 12956.67753182,
        12984.40304949, 12979.74164093, 12991.01582298, 13081.94048115,
        13040.62379754, 13003.61903261, 12894.35300059, 12946.58953575,
        12942.06638284, 13011.03209541, 12950.24100666, 12923.88234207,
        13046.3260429 , 13018.5887884 , 12908.4808072 , 12869.41165279,
        12932.74431426, 12893.85879537, 12944.58343303, 12944.31741031,
        12852.38852426, 12874.15756655, 12826.40883846, 12830.0609336 ,
        12824.19292011, 12817.51517737, 12793.6721395 , 12796.90003578,
        12784.7917796 , 12708.47722408, 12623.8740161 , 12655.0783638 ,
        12580.20594988, 12490.74390588, 12376.94213234, 12231.34960073,
        12138.28461114, 12018.4975767 , 11912.96233649, 11833.02635152,
        11775.12317664, 11732.47630256, 11704.08683556, 11683.41513624,
        11667.22704462, 11660.79590918, 11644.84200842, 11657.14447244,
        11651.18474213, 11638.09703836, 11651.05687511, 11671.63253441,
        11662.89304115, 11659.39558686, 11658.28039104, 11635.42361358,
        11633.49983445, 11627.29931245, 11617.97149313, 11605.83204774,
        11603.25543033, 11590.82973553, 11591.69916428, 11576.53706877,
        11576.67310137, 11590.70233456, 11594.88364324, 11584.83719085,
        11570.6991535 , 11573.78996346, 11564.59737407, 11549.35606338,
        11534.77579724, 11549.68950494, 11552.92527329, 11558.05553697,
        11565.50363749, 11576.02459228, 11595.17495657, 11597.74399457,
        11626.33100492, 11643.34977458, 11673.7799425 , 11691.4132004 ,
        11684.84306777, 11686.06116326, 11681.96655127, 11688.99714415,
        11691.67789221, 11683.0978767 , 11695.20245402, 11706.67775089,
        11703.88474648, 11698.08126854, 11679.90894736, 11698.92243457,
        11700.19769098, 11686.69598812, 11683.49223648, 11712.00633146,
        11699.1087357 , 11706.89071473, 11711.86991678, 11708.19620067,
        11713.23587238, 11698.58532866, 11724.6372391 , 11750.0768288 ,
        11753.78690358, 11757.9930032 , 11772.54585652, 11783.41162169,
        11792.07785284, 11763.79908379, 11772.40622973, 11780.70602115,
        11785.92471609, 11779.0009832 , 11784.451499  , 11767.59178974,
        11762.43767573, 11776.08340493, 11807.11563666, 11789.54966707,
        11805.77874496, 11810.35469709, 11810.85552692, 11848.43117857,
        11838.12098851, 11846.75675846, 11856.76896677, 11890.04328654,
        11883.28481375, 11897.50134263, 11912.21320334, 11900.77246134,
        11895.76349734, 11888.30185278, 11885.60895626, 11889.32754414,
        11886.12193332, 11905.53188321, 11908.69492352, 11927.66232882,
        11924.48302063, 11904.29668386, 11915.77696224, 11919.22582808,
        11915.18399095, 11915.67524302, 11916.6666188 , 11929.55184687,
        11924.31682974, 11933.42029016, 11986.33613784, 11979.35091182,
        11982.81310748, 11994.68448992, 12007.25599583, 12028.31352381,
        12036.70424152, 12060.83261373, 12092.81055135, 12116.281475  ,
        12131.69510645, 12155.10303526, 12162.24340999, 12147.15839093,
        12154.44237307, 12157.29306963, 12152.38072364, 12145.90807844,
        12121.30528902, 12129.75218855, 12106.05208413, 12093.98371482,
        12074.06150585, 12050.99729622, 12001.66900871, 11974.76575586,
        11956.73992859, 11940.83031465, 11909.23519912, 11900.04772006,
        11894.91874468, 11891.2470804 , 11883.04849583, 11898.29374136,
        11893.98709212, 11901.49669334, 11914.23389894, 11917.88703521,
        11920.27742695, 11921.95810329, 11913.37780715, 11907.27908649,
        11889.98411073, 11875.2596675 , 11879.48485178, 11886.93436478,
        11884.43334663, 11886.100942  , 11897.9738892 , 11899.95170549,
        11899.98850472, 11906.65766098, 11913.89659255, 11906.24617118,
        11923.63392312, 11925.13785877, 11942.96533288, 11956.28334292,
        11964.11299746, 11982.02881044, 11996.06314737, 12009.14147955,
        12017.9737942 , 12021.56894636, 12031.34558735, 12042.68036081,
        12058.57647551, 12080.34821043, 12086.26624808, 12094.44864935,
        12087.9083634 , 12082.61946736, 12084.99725962, 12075.07582969,
        12058.0711477 , 12038.08210918, 12028.71813596, 12018.05918665,
        12015.69567658, 12004.69959763, 11997.12785432, 11987.53422946,
        11986.48689267, 11985.38342653, 11981.87731684, 11969.28212952,
        11952.05774226, 11930.22654009, 11904.68055682, 11882.98298453,
        11861.72694105, 11838.82012468, 11822.18661531, 11808.48429226,
        11794.57635804, 11780.37104447, 11768.92890222, 11760.08503884,
        11758.92081632, 11752.50202572, 11749.84984541, 11747.30115676,
        11747.29268054, 11755.47823014, 11771.43980195, 11782.13641351,
        11798.91130415, 11810.0175033 , 11821.07662611, 11831.58092358,
        11840.17775517, 11844.69772942, 11847.43323711, 11846.69501179,
        11844.86466306, 11844.75869331, 11843.62075868, 11835.21274263,
        11824.30000406, 11814.91036472, 11803.63569337, 11797.86975664,
        11792.04695537, 11779.30845907, 11766.34027298, 11751.78311746,
        11739.80903651, 11723.30291529, 11710.54921831, 11711.20824808,
        11714.72740452, 11721.21750914, 11728.14236873, 11727.14978265,
        11719.81617409, 11704.56339794, 11696.79891904, 11692.01767654,
        11680.41443858, 11666.10283304, 11653.51724261, 11651.93667692,
        11646.99041062, 11636.83563449, 11629.35605449, 11621.06460702,
        11612.94704123, 11610.01082482, 11618.61716726, 11619.68833867,
        11616.45168385, 11621.61936643, 11626.35933203, 11632.29172715,
        11640.90694881, 11651.6845599 , 11662.02322878, 11674.22191095,
        11683.84896721, 11691.51492113, 11697.99947612, 11705.22458904,
        11714.31286232, 11723.43304438, 11732.51385401, 11742.69956855,
        11752.57736092, 11762.04778971, 11770.39995314, 11780.03371255,
        11790.70259034, 11797.07460945, 11804.68564134, 11811.2647658 ,
        11818.07960278, 11820.61535607, 11822.26919273, 11820.78353781,
        11820.69370569, 11820.08842433, 11819.16945762, 11820.39312312,
        11817.32353171, 11810.44071994, 11798.21870736, 11789.97762105,
        11782.24928406, 11777.89159746, 11778.01280632, 11779.48331068,
        11781.49776524, 11783.06031454, 11782.06782184, 11778.28272704,
        11770.06610314, 11765.82505569, 11765.90683257, 11764.73116078,
        11767.80152961, 11773.893752  , 11773.80261929, 11775.50612518,
        11778.19969014, 11778.48001768, 11783.62156361, 11790.42984386,
        11800.32671386, 11811.24682723, 11820.13434593, 11832.02111905,
        11841.31683172, 11850.8116081 , 11857.52285872, 11864.0518379 ,
        11873.62363175, 11880.44619923, 11887.95341106, 11895.77348726,
        11900.1704637 , 11903.40316524, 11903.43148211, 11903.19556855,
        11901.51918068, 11904.0603407 , 11906.60931247, 11910.64127439,
        11917.96416026, 11927.07481187, 11934.7478757 , 11940.09295859,
        11946.79774616, 11948.1925594 , 11943.50485647, 11932.94288319,
        11921.37375145, 11906.29825697, 11893.97659485, 11886.34545963,
        11879.94071729, 11876.37584234, 11872.08371434, 11863.92208238,
        11855.51738522, 11845.62298763, 11834.48920291, 11829.76906574,
        11828.1866995 , 11824.37191166, 11823.42277868, 11829.38706833,
        11836.52056722, 11847.00799423, 11856.57962011, 11865.34959911,
        11874.09390234, 11871.25616835, 11871.19458378, 11877.35169412,
        11878.45613019, 11878.57230654, 11887.05945769, 11902.49833306,
        11915.60915771, 11927.04163055, 11932.86889293, 11934.98253268,
        11932.15717315, 11926.82208337, 11919.44564689, 11910.84176359,
        11903.09920347, 11898.9361283 , 11893.7699388 , 11887.98415711,
        11880.55340171, 11872.74407975, 11863.78954158, 11854.72081319,
        11848.29320033, 11842.27266778, 11837.07143523, 11832.60631795,
        11828.74532121, 11826.86213186, 11823.53821641, 11822.13545507,
        11824.66444457, 11828.45927002, 11828.94467815, 11831.26508282,
        11830.33509392, 11822.48577495, 11811.48306944, 11797.76660482,
        11780.58172247, 11763.48093189, 11747.69905696, 11732.64044284,
        11719.56167746, 11708.56976383, 11702.12652161, 11698.93516681,
        11701.42501875, 11702.31315873, 11704.97451751, 11704.852263  ,
        11702.64313026, 11699.22652585, 11693.99279811, 11683.6249866 ,
        11672.49538983, 11658.80527699, 11647.05789687, 11639.37380097,
        11635.72683489, 11636.70940691, 11643.5672253 , 11653.1311714 ,
        11662.21539853, 11664.20231831, 11664.1309571 , 11661.28103011,
        11658.87279232, 11656.72140628, 11651.53309018, 11648.49922437,
        11649.78126766, 11646.44797847, 11644.33933532, 11644.35814805,
        11643.25814965, 11643.89564524, 11649.34780234, 11653.47364349,
        11661.90917225, 11670.44650463, 11676.43286094, 11677.92315596,
        11672.77137716, 11664.18191736, 11656.20242302, 11646.0292334 ,
        11637.90220288, 11632.59598486, 11629.87845218, 11629.81672525,
        11630.46428867, 11634.06550643, 11641.02994071, 11651.08944619,
        11657.60530229, 11663.11656642, 11667.1987472 , 11665.04575112,
        11660.9195405 , 11651.23704766, 11642.81256964, 11636.0308698 ,
        11633.47128264, 11632.57271556, 11631.54789911, 11630.65691821]),
 'penalty': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0.]),
 'zcoeff': array([[ 4.34404203e-01,  3.39780043e+03,  1.91495608e+01,
          1.51298654e+02],
        [ 4.33827847e-01,  3.37111840e+03,  1.91904697e+01,
          1.52249475e+02],
        [ 4.33410878e-01,  3.32411194e+03,  1.91667319e+01,
          1.52393500e+02],
        ...,
        [ 1.16385531e-01,  4.66241406e+00,  4.59409933e+00,
         -1.13803158e+00],
        [ 1.15956689e-01,  4.66888924e+00,  4.60481829e+00,
         -1.08086518e+00],
        [ 1.15493839e-01,  4.66502619e+00,  4.61922915e+00,
         -1.00983056e+00]]),
 'zfit': <Table length=0>
    z      zerr  zwarn   chi2       zz        zzchi2      coeff    npixels spectype subtype ncoeff
 float64 float64 int64 float64 float64[15] float64[15] float64[10]  int64    str6     str3  int64
 ------- ------- ----- ------- ----------- ----------- ----------- ------- -------- ------- ------}
sbailey commented 2 months ago

Thanks for the checks @moustakas . The read_zscan -> zfit problem you found is a genuine bug, but it is also present in main and is due to some parsing problem in read_zscan; the info is available in the underlying hdf5 file on disk. I'm going to merge this PR as-is and fix that problem in a separate PR.