Initial versions of spectroscopic aperture correction datamodels

[stscijgbot]

Issue JP-1049 was created by Alicia Canipe:

For spectroscopy the aperture correction file will generally be a FITS table with columns FILTER, PUPIL (NIRISS and NIRCam) or SUBARRAY (MIRI) or one of GRATING (NIRSpec IFU and MOS) or SLIT (NIRSpec fixed slit), ORDER (used only for the NIRISS WFSS mode), NELEM, WAVELENGTH, HEIGHT, APCORR, SKYIN, and SKYOUT. The latter three values are floating point arrays of length given by the integer value NELEM. Wavelengths are in microns, HEIGHT is the slit extraction width in arc-seconds analgous to the RADIUS value for imaging, and the APCORR value is a multiplicative correction factor greater than or equal to 1.0. For spectroscopy the sky background subtraction is generally assumed to be carried out prior to the spectroscopic extractions, such as using a background image or a scaled background template. Nonetheless there is the option for the SKYIN and SKYOUT values to be specified as a function of wavelength. The SKYIN and SKYOUT values are in pixels by assumption, and are measured in cross dispersion direction on both sides of the spectrum. If the two values are identical, no background subtraction needs to be carried out. [The definition of SKYIN and SKYOUT for spectroscopy requires further discussion.]

For the case of MIRI/MRS the aperture correction values will be given as a two-dimensional floating point image of dimensions 1032x1024 pixels, analogous to the photometric calibration reference file. As with the other modes the aperture correction values will be multiplicative corrections greater than or equal to 1.0.

The table below provides the formatting details and the CRDS and table row selection criteria for each instrument.

{color:#ff8b00}THIS TABLE IS NOT FINAL.{color}

!image-2019-10-14-10-21-36-088.png!

[stscijgbot]

Comment by Sarah Kendrew: for MIRI LRS we have an initial version of the aperture correction file available. Our format is different from the datamodel described above, in that we have a wavelength axis, and a 2-D array of values that define aperture corrections as a function of extraction width, at the specific wavelength point. This will allow the appropriate aperture correction to be applied when using varying-width extraction (e.g. tapered column). Can the datamodel be formatted to accommodate such a 2-D scheme? I've copied our columns scheme in below.

A few further comments:

• can we not use HEIGHT to define the extraction size? For LRS the spectrum is dispersed vertically, i.e. your HEIGHT is our WIDTH, and this gets v confusing. 
• can we accommodate different units for extraction aperture sizes? we currently have ours in pixels rather than arcsec.

 

ColDefs(

    name = 'wavelength'; format = 'E'; unit = 'micron'

    name = 'width'; format = '40B'; unit = 'pixels'

    name = 'apcorr'; format = '40E'

)

 

cc [~schreibe], [~bouwman]

[stscijgbot]

Comment by Juergen Schreiber: Sarah,

Since it is desired that apcorr is a multiplicative factor we have to take the reciprocal values of our apcorr values.

(Sorry, I am still living in the simulator world)

[stscijgbot]

Comment by Sarah Kendrew: [~schreibe] that's fine. We'll probably have to iterate a few times, so we'll just add it to the list of changes we need to make...

[stscijgbot]

Comment by Kevin Volk: Why is the HEIGHT value in arc-seconds not in pixels? Are we working from resampled images or from the distorted 2-D cut-out images? In the original 2-D cut-out images one cannot go from arc-seconds to pixels in a uniform way. Of course for the recified 2-D images one can go from pixels to arc-seconds, but for NIRISS this assumes that the grism does not produce any distortion of its own beyond what we see in imaging mode, which may not eb correct.

[stscijgbot]

Comment by Kevin Volk: Why is the HEIGHT value in arc-seconds not in pixels? Are we working from resampled images or from the distorted 2-D cut-out images? In the original 2-D cut-out images one cannot go from arc-seconds to pixels in a uniform way. Of course for the rectified 2-D images one can go from pixels to arc-seconds, but for NIRISS this assumes that the grism does not produce any distortion of its own beyond what we see in imaging mode, which may not be correct.

[stscijgbot]

Comment by Alicia Canipe: NIRISS: [~kvolk]/[~swara] NIRCam: [~hilbert]/[~npirzkal] MIRI: [~skendrew]/[~dlaw]/[~sargent] NIRSpec: [~muzerol]/[~pena] SCSB: [~hodge]/[~eisenham]

Can you please look at the information in the "Description" above and check the table to make sure the format matches your understanding from today's meeting? Once we have confirmation from all instruments that everything is okay, I will pass the baton to SCSB so they can create the datamodels for us to use to create the reference files.

[stscijgbot]

Comment by Juergen Schreiber: Alicia,

I can comment on the table entry MIRI LRS in addition what Sarah might comment.

1) For MIRI LRS we plan to have the aperture size in pixels NOT in arcsec.

2) As Sarah already commented we plan to have about 40 aperture sizes for each wavelength entry.

In your table scheme that means that the apcorr and size arrays are 2D arrays with a shape of 40 * 500. 

[stscijgbot]

Comment by Kevin Volk: The description for NIRISS WFSS matches what was discussed. Something that was not discussed is what size to recommend to the pipeline (which probably will vary by filter, but we need to think about this). I assume for NIRISS WFSS we will want one standard size that the pipeline uses automatically, and possibly other sizes to given users the option of using a different size aperture for point sources if they so wish. Another question for the SCSB people is whether there will be a way to turn off the aperture corrections? As noted at the CAL WG meeting last week users of HST WFSS are used to there being no aperture corrections applied to the WFSS spectra, and they may want the same from the JWST data. It would be good to have a flag in the extract1d step that turns off the aperture corrections, if this is not too much trouble.

[stscijgbot]

Comment by Kevin Volk: The description for NIRISS WFSS matches what was discussed. Something that was not discussed is what size to recommend to the pipeline (which probably will vary by filter, but we need to think about this). I assume for NIRISS WFSS we will want one standard size that the pipeline uses automatically, and possibly other sizes to give users the option of using a different size aperture for point sources if they so wish. Another question for the SCSB people is whether there will be a way to turn off the aperture corrections? As noted at the CAL WG meeting last week users of HST WFSS are used to there being no aperture corrections applied to the WFSS spectra, and they may want the same from the JWST data. It would be good to have a flag in the extract1d step that turns off the aperture corrections, if this is not too much trouble.

[stscijgbot]

Comment by Misty Cracraft: Filter should not be used for either MIRI LRS or MRS.

[stscijgbot]

Comment by Michael Migo Mueller: Two things as far as MIRI-MRS is concerned: 

1) I was operating under the assumption that aperture extraction will correct for residual background flux through a standard background annulus (to estimate both background flux and associated noise).  Is this not what we do?  That would require an update to the MIRI-MRS CDP (but not to its format).

2) The data model uses a circular aperture, which is what we're using for the time being.  There's a chance, however, that we will generalize that to elliptical in the future, at least for a fraction of the wavelength range.  This is because the MRS PSF is significantly elliptical at short wavelengths.  The format of the APERCORR CDP already allows for that (see tech note MIRI-DS-00011-NLC).  It might be wise to do the same thing in the data model.  The relevant columns are "a_over_b" and "pos_angle".

[stscijgbot]

Comment by David Law: Based on the table above and other conversations I've constructed a draft apcorr reference file ([^miri-apcorrpar-58892.fits]), datamodel ([^apcorr.py]), and schema ([^mirmrs_apcorr.schema.yaml]) for the MRS.  This is a FITS table with 1d vectors.  The single reference file is applicable to all MRS bands, with information selected out based on wavelength alone (TBC).  Reference file is created with https://github.com/STScI-MIRI/miri3d/blob/master/miri3d/apcorr/make_apcorr.py

I hadn't appreciated previously that these apcorr reference files are intended to be the main reference file for the extract1d step; as such, there are a few additional columns that I have added:

inner_bkg and outer_bkg: Placeholder columns for potential future implementation of a subtraction annulus (currently zeros)

axis_ratio and axis_pa: Placeholder columns for potential future implementation of elliptical apertures (currently ones and zeros)

apcorr_err: Placeholder column for future implementation of error estimates

Feedback appreciated.

 

[stscijgbot]

Comment by Maria Pena-Guerrero: NIRSpec has a meeting next Wed 19 in the morning to talk about the specifics of this. Overall we agree with the format but we are thinking about adding one or two columns. We will come up with a more definitive answer next Wed.

[stscijgbot]

Comment by Misty Cracraft: MIRI MRS files also loaded on https://jira.stsci.edu/browse/CRDS-337 which contains the MRS and LRS spectroscopic apcorr reference files for delivery.

[stscijgbot]

Comment by Juergen Schreiber: In the table there should be one small correction for MIRI LRS:

The wavelength column is 1d, so 500 would be fine (not 40x500).

There is only one wavelength for each 40 pixel slitlet

[stscijgbot]

Comment by Philip Hodge: [~dlaw] wrote "I hadn't appreciated previously that these apcorr reference files are intended to be the main reference file for the extract1d step; as such, there are a few additional columns that I have added: ..."

The extract_1d step will still need to have its own reference file, partly for backwards compatibility. It makes a lot of sense to me, though, to have extract_1d get the extraction size from the SIZE (or RADIUS, for IFU) column in the spectroscopic aperture correction reference file. This provides considerable flexibility for varying the extraction size with wavelength, as well as making it possible to specify the units (arcseconds or pixels). But what about the case where the SIZE column is 2-D, so that the corresponding APCORR column can give the aperture correction for any one of several sets of extraction sizes? In that case, extract_1d could use one of the columns (or rows, depending on programming language) in the SIZE array as the sizes to use for extraction at different wavelengths. The default column (row) could be specified by a keyword in the table header, and the user could override the default by giving (somehow) a different column (row) number within the SIZE array. The option of a 2-D array for SIZE is currently only used for MIRI LRS, but perhaps other instrument teams will want to use this option as well.

How can regions for background subtraction be specified? The draft reference file for MIRI MRS has additional columns (e.g. INNER_BKG, OUTER_BKG). Should we assume that extract_1d will never do its own background subtraction, except for IFU?

[stscijgbot]

Comment by Alicia Canipe: Thank you, [~schreibe] – I updated the table.

[stscijgbot]

Comment by Alicia Canipe: [~kvolk] / [~dlaw] can you take a look at Phil's comment above? We can figure out how to proceed based on your feedback. Thank you!

[stscijgbot]

Comment by Kevin Volk: I am not entirely sure what Phil has in mind in this comment. For the NIRISS WFSS case, that being the only one I am familiar with, the aperture corrections are provided for a range of SIZE values, all odd values from 3 to 21 pixels. I am assuming that user wants the pixel at the peak of the trace and some number of pixels on each size, hence the values are all odd numbers. The user can then select the aperture SIZE that they want for a given extraction, say 3 pixels if one has a faint source at the detection limit or a very crowded field or a larger SIZE value if the sources are fairly well separated. The aperture needs to be a parameter for extract1d, not found from the apcorr reference file. That would be especially true if the aperture was to be increased in size at longer wavelengths. On the other hand it is not clear how the SIZE as a function of wavelength would be specified to the extact1d step.

Also, I am explicitly assuming that the SIZE value is an integer, and was thinking of the step being applied to non-rectified data. If fractional SIZE values are allowed the aperture corrections could be interpolated between the values provided.

The NIRISS WFSS aperture correction currently is a one-dimensional vector, with multiple SIZE values per wavelength. It could be redone as a two-dimensional array if that would be helpful. I am not sure how one would specify the wavelengths and SIZE values corresponding to the (x,y) values in the Table.

[stscijgbot]

Comment by Kevin Volk: I am not entirely sure what [~hodge] has in mind in this comment. For the NIRISS WFSS case, that being the only one I am familiar with, the aperture corrections are provided for a range of SIZE values, all odd values from 3 to 21 pixels. I am assuming that user wants the pixel at the peak of the trace and some number of pixels on each size, hence the values are all odd numbers. The user can then select the aperture SIZE that they want for a given extraction, say 3 pixels if one has a faint source at the detection limit or a very crowded field or a larger SIZE value if the sources are fairly well separated. The aperture needs to be a parameter for extract1d, not found from the apcorr reference file. That would be especially true if the aperture was to be increased in size at longer wavelengths. On the other hand it is not clear how the SIZE as a function of wavelength would be specified to the extact1d step.

Also, I am explicitly assuming that the SIZE value is an integer, and was thinking of the step being applied to non-rectified data. If fractional SIZE values are allowed the aperture corrections could be interpolated between the values provided.

The NIRISS WFSS aperture correction currently is a one-dimensional vector, with multiple SIZE values per wavelength. It could be redone as a two-dimensional array if that would be helpful. I am not sure how one would specify the wavelengths and SIZE values corresponding to the (x,y) values in the Table.

[stscijgbot]

Comment by Alicia Canipe: Thank you, [~kvolk]!

[~hodge] would it be okay if the SIZE dimensions are different for some instruments?

[stscijgbot]

Comment by Philip Hodge: [~kvolk] I don't understand the table format that you described.  Do you have an example file that we could look at?

Were you thinking of having a separate file for every filter/pupil combination?  Would each of these files have a column of wavelengths, and for each row (i.e. for each wavelength) the size and apcorr columns would each contain an array of length 20?  This is not the format described in [~acanipe]'s table near the beginning of this issue.

[stscijgbot]

Comment by Kevin Volk: [~hodge] I have attached a text version of the WFSS aperture correction values. Each filter/grism combination has an associated table of values for the wavelength, the SIZE value in pixels, the aperture correction value, and the uncertainty in the aperture correction. This should correspond to what was described for NIRISS. Once the data model is ready the data values in the ascii file will be reformatted to a FITS table and stored as the reference file.

[stscijgbot]

Comment by Juergen Schreiber: Just to compare the NIRISS with the LRS format:

The NIRISS format is more a linear table with long columns each row with a different aperture SIZE, while LRS collects the aperture SIZE (here in 1 pixel steps) and the apcorr values in data arrays corresponding to each other which in the end leads to much less rows where each row has a defined wavelength.

I think in the end we should agree on one format.

 

 

[stscijgbot]

Comment by David Law: Replying to various parts of Phils comments above:

• At least for the MRS, retaining an extract_1d reference file shouldn't be hard.  At the moment that file (which I believe you created) doesn't contain any information about extraction radii that might conflict with what's in the apcorr file.  I believe it's more of a parameter reference file, dictating how the step should run rather than numerical values to use.
• Having both extract1d and aperture correction pull the aperture radius from the same file seems like it would be good to avoid duplication.
• I can see the use for a 2D RADIUS column as you describe, where there might (for instance) be three sets of allowable extractions and corrections at each wavelength, one for r=1, 1.5, and 2 PSF FWHM.  I don't think we want to avail ourselves of that for the MRS at present (though if we need to might a slight format adjustment to be consistent that's fine), but I think it sounds like that would work for LRS and NIRISS WFSS if I'm reading Kevin's comments correctly.  Essentially this would just be a reformatting of Kevin's ASCII table (for a given filter/grating) into a row per wavelength with a 2D table of different apertures/correction factors in each.  I think this would make the NIRISS and LRS formats the same.
• Sounds like nobody else wants additional background subtraction functionality?  MRS won't be using it either at the moment, but if we can include the columns to potentially allow it in future that would be useful.

Happy to iterate on draft files offline if that would be helpful.

[stscijgbot]

Comment by Kevin Volk: I looked at one of the MIRI example FITS tables, MIRI_FM_MIRIMAGE_P750L_APCORR_08.00.02.fits, and this is fine for NIRISS. The previous ascii table I produced just needs to be divided in a slightly different way to produce the apcorr reference file. Assuming this is the right one to look at, then we can go ahead with such a format. In fact I think the only reason that the previous NIRISS ascii file looked different is just that it was dumped out as ascii.

[stscijgbot]

Comment by Philip Hodge: Is MIRI_FM_MIRIMAGE_P750L_APCORR_08.00.02.fits in a directory that's accessible to us?  I'd like to take a look at it, if I may.

[stscijgbot]

Comment by Misty Cracraft: It's attached to ticket CRDS-337.

[stscijgbot]

Comment by Philip Hodge: This file is not described by any of the entries in the table near the top of this ticket.  Should it have been?  Would the MIRI LRS entry not suffice?

[stscijgbot]

Comment by Misty Cracraft: Looking at the LRS file, I see that we do not have the subarray column as specified in the table. Rather than having a file with twice as many rows, the files are split into slit (FULL subarray) and SLITLESSPRISM subarray files. The only other differences I see is the naming of column width rather than size and missing an apcorr_err array. Do we need to edit the table to remove the subarray column and edit our files to change the column name width to size and add an apcorr_err column?  [~skendrew]   [~schreibe]   How would you like to proceed?

[stscijgbot]

Comment by Philip Hodge: My interpretation of the table of formats (I could be wrong!) is that for MIRI LRS there should be two rows, one for subarray = "FULL" and one for subarray = "SLITLESSPRISM".  In each row, wavelength is an array of length 388 (at least for subarray = 'FULL'), nelem is a scalar with value 388 (the number of valid values in the wavelength array), size is a 2-D array of shape (40, 388) or (388, 40), depending on whether you use Fortran/IDL or Python order.  The apcorr and apcorr_err arrays have the same shape as size.  I reformatted the MIRI_FM_MIRIMAGE_P750L_APCORR_08.00.02.fits file as described above, and I'll give you a copy.  We can discuss this further, of course.

[stscijgbot]

Comment by Juergen Schreiber: Well, this is also possible, then we would work on 2D arrays and would merge FULL and SLITLESSPRISM in one file. So, fine with me.

[~cracraft], I can easily rename width to size and add an apcorr_err array, but up to now only filled with zeros.

[stscijgbot]

Comment by Jeroen Bouwman: I have a question concerning the "size" specification for the MIRI/LRS aperture correction. Is there a reason why this a 40*500 array? As we have calculated it now, there are only 40 aperture sizes and no wavelength dependency. For tapered apertures, interpolation to the right aperture will be done.  So in principle an array of 40 would do for size.

[stscijgbot]

Comment by Juergen Schreiber: Hi Jeroen,

yes, that is right. It is not necessary to repeat the same values 500 times. Looking at the ascii file of NIRISS I think the same is valid for NIRISS.

[stscijgbot]

Comment by Misty Cracraft: Phil Hodge sent a reformatted version of the MIRI LRS file, which I have attached to the ticket. It's MIRI_LRS.fits and the script to do it was reformat.py. I'll let the mode experts comment.

[stscijgbot]

Comment by Philip Hodge: [~skendrew], [~schreibe] , [~bouwman] , Do you want the option to use the SIZE (or RADIUS) to specify to extract_1d to extract with a wavelength-dependent aperture?  Several people have said they want extract_1d to have this option.  Is there agreement that you do not want it?  If that's the case, that's fine with me.

Do you want to limit the SIZE (RADIUS) to a 1-D array in order to keep the size of the apcorr reference file small?  Considering the size of our science data files, I don't think the apcorr files should be a concern.

[stscijgbot]

Comment by Juergen Schreiber: I have attached our corresponding CDP file MIRI_FM_MIRIMAGE_P750L_APCORR_8B.00.10.fits , too, now with both subarrays in one file and a more user friendly sequence of apertures.

Additionally I attached the schema file miri_aperture_correction_lrs.schema.yaml.

Here the SIZE array contains just all the integer pixel aperture sizes corresponding to the apcorr array.

The nelem entry here relates to the number of apertures, but it could also relate to the number of wavelengths, in principle doesn't matter and could be removed?

The final user applied size (might be wavelength dependent) and position of the aperture should be user specified in extract_1d. 

Does this make sense?

[stscijgbot]

Comment by Kevin Volk: I looked at MIRI_LRS.fits just to see what the format was. It looks like now there is a one-dimensional array of wavelengths and a two-dimensional array of SIZE values with one dimension being the same as the number of wavelengths. Is this intended to allow a different grid of SIZE values for each wavelength? In the case of NIRISS WFSS we have the same "sample" SIZE values for each wavelength. If one wanted the extraction size to vary with wavelength in extract_1d that would need to be defined separately from the file here. If that is to be done it would be independent of the filter/grism being used, a set function of wavelength for the WFSS mode. For myself, I have been assuming that the SIZE value should be set by the user, or a default value, in the making of the 2-D cut-outs. Once this value is set, the extraction is done on the full 2-D aperture. This extraction size might be filter-dependent but is not defined by the segmentation map, at least for point sources. For a large extended source the extraction size would need to match the segmentation map dimension, but only if that was larger than the point source extraction size. It is not clear that extracting the spectrum for a large extended source is actually of any use in WFSS mode because the wavelengths will blur along the dispersion direction. If one got lucky and had a source that was long and thin, and oriented such that the narrow dimension was in the dispersion direction then one could still get a reasonable spectrum, but of course this is unlikely in general.

For NIRISS the intent is to have a table of wavelengths, one of SIZES, and one of APCORR values for each filter. The SIZE values and wavelength values will have duplicate values and only the APCORR values will be distinct in all entries. It is not clear how to make this match the MIRI_LRS.fits format, and it seems that this would make the FITS table larger than needed...but I may be misunderstanding what is intended.

[stscijgbot]

Comment by Philip Hodge: Thank you all for your comments; this has been helpful.  It would be fine with me for SIZE to be a one-dimensional array.  It appears that the extraction size for extract_1d should be specified in the extract_1d reference file, rather than in the apcorr reference file, and this makes sense to me.  Although there's certainly room for discussion if people have other opinions.

I do need to elaborate on the NELEM column, however.  This could apply to either the array of wavelengths or the array of sizes (or radii), as you have pointed out.  The reason why it's needed is to distinguish between the size of the column of wavelengths and the number of valid elements in a given row of the wavelength column.  A case in point is [~kvolk]'s table for NIRISS WFSS.  There are 12 different combinations of filter and pupil, so there would be 12 rows in the table.  The relevant row to use will be selected based on filter and pupil.  But the number of elements in the wavelength array (and apcorr and apcorr_err) in Kevin's table can be 70, 110, or 130.  So the allocated size of the wavelength array would need to be 130 (a 130-element array for each row).  The NELEM column would specify how many of those are populated, i.e. it would have a value of either 70, 110, or 130.  It could well be the case that the number of elements in the SIZE column could differ from one row to another (though it doesn't in this particular table), so we might want two columns (e.g. NELEM_WL and NELEM_SIZE) to specify the valid numbers of elements in those two arrays, and of course in the APCORR and APCORR_ERR arrays because they depend on the number of valid elements in both SIZE and WAVELENGTH.

[stscijgbot]

Comment by Juergen Schreiber: well, or use a tuple (NELEM_WL, NELEM_SIZE)

[stscijgbot]

Comment by Alicia Canipe: Okay, so following the discussion above, it seems like we are in agreement that the SIZE column is 1D, and MIRI has combined the tables for their two subarrays into one file. Is that right? The final question is NELEM (see Phil's comment) – should we have NELEM_WL to specify the number of elements in the wavelength column and NELEM_SIZE for the SIZE column?

[~kvolk] and [~schreibe]  what do you think? I'll wait to make the final updates to the table once I know what works best for you. 

[stscijgbot]

Comment by Kevin Volk: Personally for NIRISS I think there is no pressing need for separate NELEM_SIZE and NELEM_WL parameters, and it is not clear what one would do with these. If the size of APCORR arrays is just NELEM_SIZE * NELEM_WL then does it matter whether we specify the individual ones or just NELEM? I am still not clear on how to structure the tables. For NIRISS the SIZE values are fixed for all the filters but one still needs to associate one wavelength and one SIZE value for a given APCORR value, so the SIZE array needs to be as big as the APCORR array and ditto for the WAVELENGTH array unless the APCORR array is 2-D [NELEM_SIZE, NELEM_WL]. Of course that can be done and saves a bit of space, but the tables are not all that long so it not a big issue.

[stscijgbot]

Comment by Juergen Schreiber: Kevin, as far as I understand your ASCII table, the proposal of Phil is reasonable:

We need a selection criteria to avoid having a dozen of separate CRDS files for each Filter/pupil combination, so each table row is for one filter/pupil combination, it has always the same NELEM_SIZE (number of apertures the same for each wavelength) but different NELEM_WL, so NELEM_WL is essential.

SIZE could be 1-d and apcorr 2-D, so in principle the same as for LRS.

[stscijgbot]

Comment by Kevin Volk: [~schreibe], [~hodge] I am not understanding something here. Presumably there is either one FITS table for NIRISS with all the FILTER/PUPIL/GRISM combinations or two such FITS tables, if we need one such Table per GRISM, but not a different table for each FILTER/PUPIL combination. Why would we need to have separate CRDS files for each FILTER/PUPIL combination? The ASCII table I posted a while ago just shows the values for NIRISS, it does not mean that one needs a separate FITS table for each filter. It was intended to show the run of the values, but not how these are arranged in the FITS Table.

[stscijgbot]

Comment by Alicia Canipe: Yes, I think we don't need separate files for each FILTER/PUPIL combination. If I'm understanding correctly, the issue that was still up in the air was the shape of the columns.

If we can agree on the following:

• SIZE is 1D
• If LRS needs NELEM_SIZE and NELEM_WL, then I would like to adopt that for all the instruments – if an instrument doesn't need different shapes for those two columns, then is doesn't hurt anything, but it allows LRS to have the format they need

I'm wondering if it would be easiest to solve this via a short webex session, [~schreibe] and [~kvolk]? Unless you are okay with SIZE=1D and NELEM_SIZE, NELEM_WL.

[stscijgbot]

Comment by Kevin Volk: I can arrange the values in a two-dimensional table with SIZE and WAVELENGTH along with two parameters NELEM_SIZE and NELEM_WL if that is what is needed. This should be straightforward. So if the LRS needs this and we want uniformity I can do it. I can participate in a telecon if that would be helpful. I agree that some of this back-and-forth could have been avoided, and the process would have been faster, if we had discussed it rather than posting these comments.

[stscijgbot]

Comment by Juergen Schreiber: I think the format/table is fine now for LRS

[stscijgbot]

Comment by David Law: I just attached miri-apcorrpar-58941.fits which updates the MRS reference file format to follow the convention discussed at the meeting earlier today (i.e., including nelem_wl as a column entry).

[stscijgbot]

Comment by Howard Bushouse: JP-1401 has been filed for SCSB to create the data models, as specified above.

[stscijgbot]

Comment by Howard Bushouse: Just a clarification, for those who may have started constructing their own reference tables: the values nelem_wl and nelem_size do NOT get set to the physical lengths of the wavelength and size arrays, but rather are set to the number of valid values that should be loaded from those arrays. For example, if the physical size is 500, but for a particular mode there are only 350 valid values, populate the first 350 entries of wavelength and/or size with those valid values, set the remaining 150 entries to zero, and set nelem_wl/nelem_size=350.  nelem_wl and nelem_size can have different values, if necessary, within a given table row.

The only restriction is that the wavelength array must have the same physical size in all rows of a given table, and as must size (but it can be different from wavelength).