MillionConcepts / pdr

[P]lanetary [D]ata [R]eader - A single function to read all Planetary Data System (PDS) data into Python
Other
60 stars 6 forks source link

`pdr.Data` cannot find explicit pointers nested inside PVL groups #15

Closed m-stclair closed 3 years ago

m-stclair commented 3 years ago

for instance, Data finds this pointer:

^IMAGE = ("2259ML0120880010804894C00_DRCL.IMG")

but not this pointer:

GROUP                              = GEOMETRIC_CAMERA_MODEL_PARMS
 ^MODEL_DESC                         = "GEOMETRIC_CM.TXT"

or this pointer:

OBJECT                  = L0_FILE                                             
  ^L0_LINE_PREFIX_TABLE = "M3G20081124T020103_V01_L0.IMG"  

resolution might require recursively iterating over PVLModule.values() or an entirely different approach.

m-stclair commented 3 years ago

stopgap resolution in c2c4b4407e08f0653113def25e9cc49f3df5cff7, requires more testing

m-stclair commented 3 years ago

and a little more aggressively: a219523

michaelaye commented 3 years ago

i wanted to share this nicely written post about general purpose recursive structure digging. I found it very clear, clean, and simple (which doesn't happen often for recursive code for me) : http://nvie.com/posts/modifying-deeply-nested-structures/

michaelaye commented 3 years ago

oh, and i do this simple thing for index tables, where the pointers are also at different places sometimes, not sure it's applicable/useful here:

https://github.com/michaelaye/nbplanetary/blob/master/planetarypy/pds/indexes.py#L114-L118

m-stclair commented 3 years ago

the 'general purpose recursive structure digging' technique is very close to how i solved this problem previously, and is good for tracking cases in which there are PVL 'file area' blocks one level up from the PVL block that contains an actual object description. the question is whether we want to have Data's pointers be sequences -- and I suppose we probably do. currently as a stopgap I am using, basically, a truncated version of that:

def dig_for(mapping, target, predicate=eq):
    nests = [v for v in mapping.values() if isinstance(v, Mapping)]
    level_keys = [(k, v) for k, v in mapping.items() if predicate(k, target)]
    if nests:
        level_keys += reduce(
            add, [dig_for(nest, target, predicate) for nest in nests]
        )
    return level_keys

def dig_for_value(mapping, target, predicate=eq):
    dug = dig_for(mapping, target, predicate)
    if dug:
        return dug[0][1]
    return None

...and then as methods of Data:

    def labelget(self, text):
        return dig_for_value(self.LABEL, text)

    def labelblock(self, text):
        what_got_dug = dig_for_value(self.LABEL, text)
        if not isinstance(what_got_dug, Mapping):
            return self.LABEL
        return what_got_dug
m-stclair commented 3 years ago

...and to grab all pointers:

def get_pds3_pointers(label=None, local_path=None):
    if label is None:
        label = pvl.load(local_path)
    return dig_for(label, "^", lambda k, v: k.startswith(v))
m-stclair commented 3 years ago

closing this for now with these stopgap solutions. see #21 for continued work.