spectriclabs / elastic_datashader

:earth_americas: Datashader enabled TMS server with ElasticSearch backend
Apache License 2.0
4 stars 1 forks source link

Add tests for current filter types and refactor filter creation #44

Closed desean1625 closed 7 months ago

desean1625 commented 8 months ago

There will need to be some manual testing but I think I got all the various filters and test added before the refactor.

Need to double check filters created from controls I think that is what this code block was from

                if filter_key == "query":
                    filt_index = list(f.get(filter_key))[0]
                    filter_dict["must_not"].append({filt_index: f.get(filter_key).get(filt_index)})

when the object nested under query wasn't bool and the key wasn't provided

desean1625 commented 8 months ago

@natebynum20 This is ready for review and merge. It fixes three issues.

  1. not all filter cases where being parsed correctly
  2. if a bad filter was sent like
    {
    "exits": {
    "field": "name"
    }
    }

    the datashader would immediately have the client try again causing valid requests to get interrupted when gunicorn shuts down the worker after x number of requests to clean up memory leaks.

  3. the cleanup cache code wasn't cleaning up empty directories this would lead to a build up of directories that would eventually cause glob to take longer than the default gunicorn timeout eventually putting datashader in a state where it wouldn't start up (this would take a month or two)
BenShoeSpectric commented 7 months ago

Nate, Did you have a chance to double check the commits after you approval?

natebynum20 commented 7 months ago

Just went over the commits that were submitted after my initial approval. Looks good from my stand point