maggienj / ActiveData

Provide high speed filtering and aggregation over data
Mozilla Public License 2.0
0 stars 0 forks source link

fix - test_jx.test_groupby_1.TestgroupBy1.test_groupby_object #50

Closed maggienj closed 7 years ago

maggienj commented 7 years ago

fix unit test, test_jx.test_groupby_1.TestgroupBy1.test_groupby_object

maggienj commented 7 years ago

This is how the currently generated query looks like. I guess, i will have to remove the word "filter" , since the word "query" is already present in it. (if it weren't present then i would have replaced the word "filter" to "query"..... )

bool-must_not is already in place... so , no changes in it.

    {
        "aggs": {
            "_match": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"query": {"bool": {"must_not": {"field": "g.v"}}}}}
                },
                "terms": {
                    "field": "g.a",
                    "size": 10
                }
            },
            "_missing": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"query": {"bool": {"must_not": {"field": "g.v"}}}}}
                },
                "filter": {"query": {"bool": {"must_not": {"field": "g.a"}}}}
            }
        },
        "size": 0
    }
maggienj commented 7 years ago

well, changed the query now., removed the phrase "query" in it. so, it looks like as shown below...


{
        "aggs": {
            "_match": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"bool": {"must_not": {"field": "g.v"}}}}
                },
                "terms": {
                    "field": "g.a",
                    "size": 10
                }
            },
            "_missing": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"bool": {"must_not": {"field": "g.v"}}}}
                },
                "filter": {"bool": {"must_not": {"field": "g.a"}}}
            }
        },
        "size": 0
    }
maggienj commented 7 years ago

if i change "filter" to "query" then the last line removes the filter too.... and this appears to be an agg filter.... so, i guess the last occurrence of filter shouldn't be touched and kept as is.... "filter": {"bool": {"must_not": {"field": "g.a"}}}

only the "_missing" line should be touched.... to replace the word "filter" with "query"... but, when i try to replace the word "filter" to "query" the last line also gets changed...

well, how should this query look like for it to work correctly? should the last occurrence of "filter" be changed to "query" or kept as is?

klahnakoski commented 7 years ago

please make a branch, push your changes, and make a pull request so I may see your code.

klahnakoski commented 7 years ago

This query looks fine:

"_missing": {"filter": {"bool": {"must_not": {"field": "g.v"}}}}

_missing is a filter aggregation that accepts a bool.must_not where expression,

klahnakoski commented 7 years ago

it appears the where expression is wrong. maybe it should read

 {"bool": {"must_not": {"exists": {"field": "g.v"}}}}

for clarity the whole line should read:

"_missing": {"filter": {"bool": {"must_not": {"exists": {"field": "g.v"}}}}}
maggienj commented 7 years ago

yay! that's it! that worked! test passed! pushing code now.

maggienj commented 7 years ago

just to add this to the closing notes here... the below query works fine in es5. test passed.


    {
        "aggs": {
            "_match": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"bool": {"must_not": {"exists": {"field": "g.v"}}}}}
                },
                "terms": {
                    "field": "g.a",
                    "size": 10
                }
            },
            "_missing": {
                "aggs": {
                    "_match": {"terms": {
                        "field": "g.v",
                        "size": 10
                    }},
                    "_missing": {"filter": {"bool": {"must_not": {"exists": {"field": "g.v"}}}}}
                },
                "filter": {"bool": {"must_not": {"exists": {"field": "g.a"}}}}
            }
        },
        "size": 0
    }
klahnakoski commented 7 years ago

found a problem line:

Log.note("frum {{data}}", data=frum)

you can not log frum its structure is to complicated. Pull from upstream es5 to get a fix that shows you file and line the problem is in

maggienj commented 7 years ago

those lines were removed as part of a different branch. but it wasn't merged in this branch. since they both are two different changes going into the same es5.... left it as unmerged.

just merged es5 into this branch now... so this prob line should be gone now..

maggienj commented 7 years ago

merged. closing this. https://github.com/klahnakoski/ActiveData/pull/36