Open maggienj opened 7 years ago
some reference notes follow.... What does decoder.py do? When a query is sent to ES, a particular type of response is expected. For example, if a terms aggregation is sent, then we expect a response with "buckets" and each bucket to have a "key". Each of those bucket keys are mapped to parts of an edge in the ActiveData response. There is one decoder for each type of edge (or groupby) we send to ActiveData: The decoder is responsible to making the correct ES query, and also interpreting the ES response for building the ActiveData response.
The easiest decoder to understand is the SetDecoder. Find a test with an edge with a specific domain (a domain with a set of values): run through the decoder to see what it does; both with making the proper ES query, but also building the response.
try to document the decoder.py while going through TestEdge1.test_no_select; which is the best example of what decoders.py does.
few pages drawn and scanned.
I'm trying it figure out why we need this lambda function here... I understand that lambda is a kind of a nameless function ...
but, couldn't get it ... in this context below ....why lambda is required?
class ObjectDecoder(AggsDecoder):
def __init__(self, edge, query, limit):
AggsDecoder.__init__(self, edge, query, limit)
if isinstance(edge.value, LeavesOp):
prefix = edge.value.term.var
flatter = lambda k: literal_field(relative_field(k, prefix))
else:
prefix = edge.value.var
flatter = lambda k: relative_field(k, prefix)
self.put, self.fields = zip(*[
(flatter(c.names["."]), c.es_column)
for c in query.frum.schema.leaves(prefix)
])
Is it possible to rewrite the same code, for the sake of understanding without lambda ? please let me know how this would look like without lambda. Thanks!
I'm using the same test, which is Test_Edge_1.TestEdge1.test_no_select
and in the first pass through the lambda function which is specified in the "else" part, gets executed.
the edge variable is "a" for this test.
so, in the debugger, when we drill down the values in func_closures for this function... the cellvalue is "a"
flatter = lambda k: literal_field(relative_field(k, prefix))
<function
This test's query looks for only one column(edge) which is "a" and the above function closure shows "a"
Here is the same code without the lambda: Where ever I see flatter
being called, I can inline it.
class ObjectDecoder(AggsDecoder):
def __init__(self, edge, query, limit):
AggsDecoder.__init__(self, edge, query, limit)
if isinstance(edge.value, LeavesOp):
prefix = edge.value.term.var
self.put, self.fields = zip(*[
(literal_field(relative_field(c.names["."], prefix)), c.es_column)
for c in query.frum.schema.leaves(prefix)
])
else:
prefix = edge.value.var
self.put, self.fields = zip(*[
(relative_field(c.names["."], prefix), c.es_column)
for c in query.frum.schema.leaves(prefix)
])
The ObjectDecoder
is expecting a simple expression (a variable, or a {"leaves": variable}
) which will result in multiple primitive columns being pulled from the database. This code is responsible for listing all the fields, and giving them a put
name.
https://github.com/klahnakoski/ActiveData/blob/dev/docs/jx_clause_select.md#selecting-leaves-
Got it! Thank you!
Draw the flow chart for decoders.py.
Just draw the flow for format_list and this is sufficient as it will get repeated for other formats. ( Do not need to draw for format_table and format_cube , as it is the same flow... )
Different methods / functions exist in Decoders.py
draw each and every method flow....(1) what do they accept as input parameter (2) what do they return and (3) what was processed in between from the input to the output.
In other words, for each and every method / function draw the flow...