xpdAcq / rapidz

Fork of Streamz
https://xpdacq.github.io/rapidz
BSD 3-Clause "New" or "Revised" License
5 stars 7 forks source link

mutating linker #19

Open CJ-Wright opened 5 years ago

CJ-Wright commented 5 years ago

xref: https://github.com/xpdAcq/xpdtools/pull/62#issuecomment-443900675

This produces an interesting pair of problems for the linker.

Since we have the qoi arg the pipeline the linker will look for a qoi upstream node to subscribe to. This node might not exist, or worse we may want to use this on multiple upstream nodes. I'm not certain how to handle this properly currently. One approach would be to modify the namespace dict before adding the tomography nodes. This would allow us to make a temporary qoi node which is the same as an existing node. Sine we may use the tomo pipeline chunk multiple times we will need to modify the linker to make unique names for all the output names. This way we can keep them in the namespace without clashing.

def abs_chunk(img):
    abs = img.map(np.sum)
    return locals()

def tomo_chunk(qoi, ...):
    tomo = qoi.map(...)
    return locals()

def mutating_linker(func, namespace, input_lut, output_lut):
    # takes valid node name to qoi (abs -> qoi)
    for k, v in input_lut.items():
        namespace[k] = namespace[v]
    loc = func(**namespace)
    # takes output name and changes it (tomo -> abs_tomo)
    for k, v in output_lut.items():
        loc[v] = loc.pop(k)
    namespace.update(loc)
    # Note that this means that input_lut keys can't be a valid output node name
    # (qoi -> N/A)
    for k, v in input_lut.items():
        namespace.pop(k)