Allowing optional filters

[MadinaB]

I have been working on this trying to solve this task given in here

we'd need to build the query dynamically (depending on whether or not a suite was passed in). This is a problem that ADR could try to solve more generally 

This update helps to create query with any optional filters which user can either pass or not pass. At first, general query should be created and saved in 'some_query_name'.query file which contains all possible filters. Then optional filters (getting rid of which will not affect the code) should be saved in optional_arguments and passed to run_query as:

run_query('some_query_name', config, optional_arguments=optional_arguments, context)

So the code will update the query by deleting filters which rely on arguments which were not passed

[MadinaB]

adr skipped_tests --from today-week

optional_arguments:
['suite', 'platform']

Query before update:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': {'$eval': 'suite'}}}, {'eq': {'build.platform': {'$eval': 'platform'}}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': {'$eval': 'from_date'}}}}, {'lte': {'run.stats.end_time': {'date': {'$eval': 'to_date'}}}}]}, 'limit': 100, 'format': 'table'}

Query after update:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': {'$eval': 'from_date'}}}}, {'lte': {'run.stats.end_time': {'date': {'$eval': 'to_date'}}}}]}, 'limit': 100, 'format': 'table'}

Query after render:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': 'today-week'}}}, {'lte': {'run.stats.end_time': {'date': 'eod'}}}]}, 'limit': 100, 'format': 'table'}

[MadinaB]

adr skipped_tests --from today-week --suite xpcshell --platform maxosx64

optional_arguments:
['suite', 'platform']

Query before update:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': {'$eval': 'suite'}}}, {'eq': {'build.platform': {'$eval': 'platform'}}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': {'$eval': 'from_date'}}}}, {'lte': {'run.stats.end_time': {'date': {'$eval': 'to_date'}}}}]}, 'limit': 100, 'format': 'table'}

Query after update:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': {'$eval': 'suite'}}}, {'eq': {'build.platform': {'$eval': 'platform'}}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': {'$eval': 'from_date'}}}}, {'lte': {'run.stats.end_time': {'date': {'$eval': 'to_date'}}}}]}, 'limit': 100, 'format': 'table'}

Query after render:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': 'xpcshell'}}, {'eq': {'build.platform': 'maxosx64'}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': 'today-week'}}}, {'lte': {'run.stats.end_time': {'date': 'eod'}}}]}, 'limit': 100, 'format': 'table'}

[MadinaB]

Upper snippets is an output result after updating run_query in query.py

    for query in load_query(name):
        ....
        print("Query before update:")
        print(query)
        update_query(query, context, optional_arguments)
        print("Query after update:")
        print(query)
        query = jsone.render(query, context)
        print("Query after render:")
        print(query)
        ....

They show how does query provided in PR #69 change via passed filters

On the contrary without this PR, PR #69 does not change and passes None for filters which are not present:

adr skipped_tests --from today-week

Query before update:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': {'$eval': 'suite'}}}, {'eq': {'build.platform': {'$eval': 'platform'}}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': {'$eval': 'from_date'}}}}, {'lte': {'run.stats.end_time': {'date': {'$eval': 'to_date'}}}}]}, 'limit': 100, 'format': 'table'}

Query after render:
{'from': 'unittest', 'select': [{'aggregate': 'count'}], 'groupby': [{'value': 'result.test'}, {'value': 'run.suite.fullname'}], 'where': {'and': [{'eq': {'run.suite.fullname': None}}, {'eq': {'build.platform': None}}, {'eq': {'repo.branch.name': 'mozilla-central'}}, {'eq': {'result.status': 'SKIP'}}, {'gte': {'run.stats.end_time': {'date': 'today-week'}}}, {'lte': {'run.stats.end_time': {'date': 'eod'}}}]}, 'limit': 100, 'format': 'table'}

[ahal]

Thanks for your work here! I believe this issue has been resolved another way (plus it has a lot of bitrot), so closing. Thanks for your understanding.