Handle PDS3 Data Objects in a more general manner

[godber]

Right now we handle a single data object ^IMAGE. However, a PDS label can define multiple data objects. We should continue to expose these as attributes but also keep track of them as data objects so that a list of data objects can be returned. Note that the Data Object Pointers are paired with Data Object definitions

See the PDS Ref sections:

• 5.3.3 - Data Object Pointers
• 5.3.6 - Data Object Definitions

[godber]

This data product is probably a good example:

http://hirise.lpl.arizona.edu/PDS/EDR/ESP/ORB_030100_030199/ESP_030168_1755/ESP_030168_1755_BG12_0.IMG

It contains the following Data Object Pointers:

/* Locations of Data Objects in the file.                    */
/*     >>> CAUTION <<< The first byte is location 1 (not 0)! */

LABEL_RECORDS                  = 32768 <BYTES>
^SCIENCE_CHANNEL_TABLE         = 32769 <BYTES>
^LOOKUP_TABLE                  = 33569 <BYTES>
^CPMM_ENGINEERING_TABLE        = 49953 <BYTES>
^CALIBRATION_LINE_PREFIX_TABLE = 50013 <BYTES>
^CALIBRATION_LINE_SUFFIX_TABLE = 50013 <BYTES>
^CALIBRATION_IMAGE             = 50013 <BYTES>
^LINE_PREFIX_TABLE             = 101337 <BYTES>
^LINE_SUFFIX_TABLE             = 101337 <BYTES>
^IMAGE                         = 101337 <BYTES>
^GAP_TABLE                     = 16481337 <BYTES>

I am not sure why some of these have the same offsets. Finding a more straight forward example to start with might be a good idea. I think there are some THEMIS products that have just a couple pointers.

[cmillion]

Gah! That's too many things jammed into one file.

This is also addressing the question posed in #22, but: They should theoretically be automatically parseable with information just in the header, right? I propose we try write it so that they are all automatically parsed into a dict() or pandas-like table where the keys are equal to the LABEL_RECORDS values. So, pseudo-coded:

data = readpds('file.img')
print data.keys()
['SCIENCE_CHANNEL_TABLE', 'LOOKUP_TABLE', 'CPMM_ENGINEERING_TABLE', etc...

Almost everybody is probably just going to be interested in data['IMAGE']. But everything else is still "first order retrievable."

[godber]

I think that interface makes sense. It doesn't preserve the order, however, though the label will still contain the object order information. We can then "promote" the ^IMAGE object by adding a property so it is still easily accessible: data['IMAGE'] is equal to data.image.

EDIT: wording

[godber]

Thinking out loud after the work I did over this weekend:

• put data objects in an ordered dictionary attribute called data_objects
• implement a DataObject class that has data and image attributes like our current data and image attributes
• PDS3Image.image will be a property that points to the image attribute on the ^IMAGE data object or Null if that data object doesn't exist.

[wtolson]

For reference, these seem to be the docs for pds object types: https://pds.nasa.gov/documents/sr/AppendixA.pdf. Also for isis cube files, the documentation is pretty lacking but I've seen at least table objects in the wild (https://dl.dropboxusercontent.com/u/274828/lub3994m.342.lev1.cub).

[godber]

@wtolson can an ISIS cube file contain MULTIPLE objects like PDS?

[wtolson]

I've not seen multiple images but I have seen one image and multiple tables in the same file (see example above).