Draw the flow chart for decoders.py

[maggienj]

Draw the flow chart for decoders.py.

• Just draw the flow for format_list and this is sufficient as it will get repeated for other formats. ( Do not need to draw for format_table and format_cube , as it is the same flow... )

• Different methods / functions exist in Decoders.py

• draw each and every method flow....(1) what do they accept as input parameter (2) what do they return and (3) what was processed in between from the input to the output.

In other words, for each and every method / function draw the flow...

• input
• process
• output

[maggienj]

some reference notes follow.... What does decoder.py do? When a query is sent to ES, a particular type of response is expected. For example, if a terms aggregation is sent, then we expect a response with "buckets" and each bucket to have a "key". Each of those bucket keys are mapped to parts of an edge in the ActiveData response. There is one decoder for each type of edge (or groupby) we send to ActiveData: The decoder is responsible to making the correct ES query, and also interpreting the ES response for building the ActiveData response.

The easiest decoder to understand is the SetDecoder. Find a test with an edge with a specific domain (a domain with a set of values): run through the decoder to see what it does; both with making the proper ES query, but also building the response.

[maggienj]

try to document the decoder.py while going through TestEdge1.test_no_select; which is the best example of what decoders.py does.

[maggienj]

few pages drawn and scanned.

[maggienj]

I'm trying it figure out why we need this lambda function here... I understand that lambda is a kind of a nameless function ...

but, couldn't get it ... in this context below ....why lambda is required?

class ObjectDecoder(AggsDecoder):
    def __init__(self, edge, query, limit):
        AggsDecoder.__init__(self, edge, query, limit)
        if isinstance(edge.value, LeavesOp):
            prefix = edge.value.term.var
            flatter = lambda k: literal_field(relative_field(k, prefix))
        else:
            prefix = edge.value.var
            flatter = lambda k: relative_field(k, prefix)

        self.put, self.fields = zip(*[
            (flatter(c.names["."]), c.es_column)
            for c in query.frum.schema.leaves(prefix)
        ])

Is it possible to rewrite the same code, for the sake of understanding without lambda ? please let me know how this would look like without lambda. Thanks!

I'm using the same test, which is Test_Edge_1.TestEdge1.test_no_select

and in the first pass through the lambda function which is specified in the "else" part, gets executed.

the edge variable is "a" for this test.

so, in the debugger, when we drill down the values in func_closures for this function... the cellvalue is "a" flatter = lambda k: literal_field(relative_field(k, prefix)) <function at 0x000000000694E4A8> <type 'tuple'>: (<cell at 0x0000000006B48558: unicode object at 0x0000000005EDFCC0>,) u'a'

This test's query looks for only one column(edge) which is "a" and the above function closure shows "a"

[klahnakoski]

Here is the same code without the lambda: Where ever I see flatter being called, I can inline it.

class ObjectDecoder(AggsDecoder):
    def __init__(self, edge, query, limit):
        AggsDecoder.__init__(self, edge, query, limit)
        if isinstance(edge.value, LeavesOp):
            prefix = edge.value.term.var
            self.put, self.fields = zip(*[
                (literal_field(relative_field(c.names["."], prefix)), c.es_column)
                for c in query.frum.schema.leaves(prefix)
            ])
        else:
            prefix = edge.value.var
            self.put, self.fields = zip(*[
                (relative_field(c.names["."], prefix), c.es_column)
                for c in query.frum.schema.leaves(prefix)
            ])

[klahnakoski]

The ObjectDecoder is expecting a simple expression (a variable, or a {"leaves": variable}) which will result in multiple primitive columns being pulled from the database. This code is responsible for listing all the fields, and giving them a put name.

https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_clause_select.md#selecting-leaves-

[maggienj]

Got it! Thank you!