search

MillionConcepts / pdr

[P]lanetary [D]ata [R]eader - A single function to read all Planetary Data System (PDS) data into Python
Other
60 stars 6 forks source link

support for suffix items in PDS3 THEMIS IR #30

Open thareUSGS opened 2 years ago

thareUSGS commented 2 years ago

I'm not sure if you plan to support suffix items within PDS3 labels. These are based on the original ISIS2 capability to support side-planes, top-planes, bottom-planes, etc.

https://viewer.mars.asu.edu//planetview/inst/themis/I14818018#P=I14818018&T=2 (download the RDR).

A good figure to show an idealized example is here -- see Section 2.3.2 of this SDPSIS https://static.mars.asu.edu/pds/ODTSDP_v1/document/sdpsis.pdf#page=11 but it doesn't show the data is actually interleaved with the main "SPECTRAL_QUBE" array. For the suffix data, I guess is just becomes a simple array (or maybe just a binary table).

Note the ASU Davinci routine _loadpds supports suffix loads: http://davinci.asu.edu/index.php?title=load_pds Not sure if source code is available for Davinci?

FYI, support for these types of interleaved data was removed from the ISIS3+ format and no longer allowed in PDS4. This type of data will need to fixed (pull-apart) for PDS4 migration. Not sure how this will look (e.g. multiple arrays, main array with table, ...)?

cmillion commented 2 years ago

Thanks for the description of the issue and example data. We do plan to support these, but we haven't decided how to prioritize it. Do you have an estimate for how many data sets contain this data format?

m-stclair commented 2 years ago

We have limited support for some categories of interleaved prefix / suffix data (e.g., the M3 L0 line prefix tables), but handling multiple prefix / suffix axes is definitely a little more complicated. So far, we have chosen to split prefix/suffix data into separate objects according to whatever name is given for the prefix data in the label -- like, here, we'd presumably just split this qube into a 320x1 32-bit float array named HORIZONTAL_DESTRIPE, a 7184x1 32-bit float array named VERTICAL_DESTRIPE, and a 320x7184x10 16-bit integer array named SPECTRAL_QUBE (or maybe CALIBRATED_SPECTRAL_RADIANCE).

The Davinci source is available; we can look and see if it might provide a useful reference implementation.

m-stclair commented 2 years ago

And re: PDS4 migration, here's the specification for how we handled converting the M3 L0 images to PDS4 products: https://github.com/MillionConcepts/m3-conversion/blob/main/readmes/data_readme.md#l0

m-stclair commented 1 year ago

brief note that we've started adding support for THEMIS qubes (see, e.g. f374ead2590cd1a5fa53f816700abad45a6db7ad). Full support should be coming pretty soon