maggienj / ActiveData

Provide high speed filtering and aggregation over data
Mozilla Public License 2.0
0 stars 0 forks source link

fix - test_edge_using_missing_between #53

Closed maggienj closed 7 years ago

maggienj commented 7 years ago

fix unittest tests.test_jx.test_edge_2.TestEdge2.test_edge_using_missing_between

maggienj commented 7 years ago

caused by

    ERROR: 'Variable' object has no attribute 'to_ruby'
    File "C:\Users\user\PycharmProjects\ActiveData\jx_base\expressions.py", line 1466, in missing
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\es52\decoders.py", line 480, in append_query
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\es52\aggs.py", line 330, in es_aggsop
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\jx_usingES.py", line 158, in query
maggienj commented 7 years ago

may need to change these lines to "to_painless()"instead of to_ruby() These were changed earlier.... so not sure why these are popping up here.... may be because of the split of jx_base and jx_elasticsearch for the same file expressions.py

since, this belongs to jx_base, not sure , if I can touch this file... as possibly another developer is also using it now... ( just read that github has introduced a new feature called "codeowners"... not sure, if the purpose is one and the same..... as we have split jx_base/expressions.py and jx_elasticsearch/expressions.py for different code ownership purposes... but just stumbled upon this today... https://github.com/blog/2392-introducing-code-owners )

jx_base\expressions.py line#1465

 def missing(self):
        value = self.value.to_ruby(not_null=True)
        prefix = self.prefix.to_ruby()
        len_prefix = "("+prefix+").length()"
        suffix = self.suffix.to_ruby()
maggienj commented 7 years ago

Or should the import statement be changed ? Just direct the import towards jx_elasticsearch/expressions.py instead of whatever it is referring to now..... then we may not need to touch this file.

maggienj commented 7 years ago

I remember that we replaced all these "or"s to "bool" "should" and "and"s to "bool" "must". If so, then this would have generated the query without "or" and "and".... instead with their bool es5 equivalents.

Why is still "or" and "and" showing up.....instead of their es5 equivalents? Is it because of some file being overwritten or is it because of missed out instances of replacement from "or" and "and", to "bool" "should" and "bool" "must"

                    "filter": {"or": [
                        {"and": [
                            {"script": {"script": {
                                "inline": "(doc[\"url\"].empty) || (doc[\"url\"].value.indexOf(\"https://hg.mozilla.org/\")==-1) || (doc[\"url\"].value.indexOf(\"/\", doc[\"url\"].value.indexOf(\"https://hg.mozilla.org/\")+(\"https://hg.mozilla.org/\").length())==-1)",
                                "lang": "painless"
                            }}},
                            {"bool": {"must_not": {"exists": {"field": "url"}}}}
                        ]},
                        {"and": [
                            {"not": {"script": {"script": {
                                "inline": "(doc[\"url\"].empty) || (doc[\"url\"].value.indexOf(\"https://hg.mozilla.org/\")==-1) || (doc[\"url\"].value.indexOf(\"/\", doc[\"url\"].value.indexOf(\"https://hg.mozilla.org/\")+(\"https://hg.mozilla.org/\").length())==-1)",
                                "lang": "painless"
                            }}}},
                            {"match_all": {}}
                        ]}
maggienj commented 7 years ago

created a new branch so that certain files remain untouched...

will have to touch this file... as it raises to_ruby err... File "C:\Users\user\PycharmProjects\ActiveData\jx_base\expressions.py", line 1466, in missing

if this file's purpose is different, then how can we refactor it in such a way that all to_ruby or to_painless reside in one place and not in jx_base/expressions.py?

In addition to the above, if we change to_ruby to to_painless in the above places in jx_base, then we may loose compatibility with es14. Because, I guess , es5 may accept this to_painless...... but es14 may not accept this in this jx_base\file because, jx_base is kind of a base to es14 and es52 .... so we should not be touching jx_base files. is this right?

may be, we have to move the references of methods related to to_ruby / to_painful from jx_base/expressions.py to a different file and then we may need to test it for es14 and for es52... so that the directives and the to_ruby and to_painless are in place and compatible with es14 and es52...

just trying to figure this out...

caused by

ERROR: 'Variable' object has no attribute 'to_ruby'
    File "C:\Users\user\PycharmProjects\ActiveData\jx_base\expressions.py", line 1466, in missing
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\es52\decoders.py", line 480, in append_query
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\es52\aggs.py", line 330, in es_aggsop
    File "C:\Users\user\PycharmProjects\ActiveData\jx_elasticsearch\jx_usingES.py", line 158, in query
    File "C:\Users\user\PycharmProjects\ActiveData\jx_python\jx.py", line 73, in run
    File "C:\Users\user\PycharmProjects\ActiveData\active_data\actions\jx.py", line 62, in jx_query
    File "C:\Users\user\PycharmProjects\ActiveData\active_data\__init__.py", line 54, in output
maggienj commented 7 years ago

pull request is complete. merged. closing this. https://github.com/klahnakoski/ActiveData/pull/44