filter extension: temporal intersection not supported

[StijnCaerts]

Describe the bug The temporal intersection queries listed in the documentation of the filter API extension do not work. For the cql2-text query, you get a parsing error as pygeofilter seems to expect a different syntax:

• filter extension documentation: T_INTERSECTS(datetime, INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00:00Z'))
• compatible with pygeofilter: datetime T_INTERSECTS INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00:00Z')

This results in the following error:

[18/Jan/2024:08:12:12 +0000] 172.17.0.1:57838 "GET /search?filter=T_INTERSECTS%28datetime%2C+INTERVAL%28%272020-11-11T00%3A00%3A00Z%27%2C+%272020-11-12T00%3A00%3A00Z%27%29%29&filter-lang=cql2-text HTTP/1.0" 500 - 14
Unexpected token Token('$END', '') at line 1, column 80.
Expected one of: 
    * T_BEFORE
    * LIKE
    * T_ENDEDBY
    * GTE
    * T_DURING
    * __ANON_0
    * T_INTERSECTS
    * __ANON_2
    * T_BEGUNBY
    * IS
    * T_OVERLAPPEDBY
    * __ANON_1
    * IN
    * LTE
    * NE
    * EQ
    * GT
    * T_CONTAINS
    * T_EQUALS
    * T_METBY
    * BETWEEN
    * T_OVERLAPS
    * MORETHAN
    * T_BEGINS
    * T_AFTER
    * EQUAL
    * __ANON_3
    * T_ENDS
    * LESSTHAN
    * T_MEETS
    * LT
NoneType: None
2024-01-18 08:12:12,868 - uvicorn.error - ERROR - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 126, in feed_token
    action, arg = states[state][token.type]
KeyError: '$END'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/asgi_logger/middleware.py", line 61, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/asgi_logger/middleware.py", line 58, in __call__
    await self.app(scope, receive, inner_send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/authentication.py", line 48, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/api/middleware.py", line 75, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/brotli_asgi/__init__.py", line 87, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/api/routes.py", line 57, in _endpoint
    await func(request=request, **request_data.kwargs()), response_class
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/elasticsearch/core.py", line 384, in get_search
    base_args["filter"] = orjson.loads(to_cql2(parse_cql2_text(filter)))
  File "/usr/local/lib/python3.10/site-packages/pygeofilter/parsers/cql2_text/parser.py", line 211, in parse
    return parser.parse(cql_text)
  File "/usr/local/lib/python3.10/site-packages/lark/lark.py", line 581, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
  File "/usr/local/lib/python3.10/site-packages/lark/parser_frontends.py", line 106, in parse
    return self.parser.parse(stream, chosen_start, **kw)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 41, in parse
    return self.parser.parse(lexer, start)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 171, in parse
    return self.parse_from_state(parser_state)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 188, in parse_from_state
    raise e
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 182, in parse_from_state
    return state.feed_token(end_token, True)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 129, in feed_token
    raise UnexpectedToken(token, expected, state=self, interactive_parser=None)
lark.exceptions.UnexpectedToken: Unexpected token Token('$END', '') at line 1, column 80.
Expected one of: 
    * T_BEFORE
    * LIKE
    * T_ENDEDBY
    * GTE
    * T_DURING
    * __ANON_0
    * T_INTERSECTS
    * __ANON_2
    * T_BEGUNBY
    * IS
    * T_OVERLAPPEDBY
    * __ANON_1
    * IN
    * LTE
    * NE
    * EQ
    * GT
    * T_CONTAINS
    * T_EQUALS
    * T_METBY
    * BETWEEN
    * T_OVERLAPS
    * MORETHAN
    * T_BEGINS
    * T_AFTER
    * EQUAL
    * __ANON_3
    * T_ENDS
    * LESSTHAN
    * T_MEETS
    * LT

But also the cql2-json query does not work:

{
  "detail": "Error with cql2_json filter: 25 validation errors for Clause\nop\n  value is not a valid enumeration member; permitted: 'and', 'or', 'not' (type=type_error.enum; enum_values=[<LogicalOp._and: 'and'>, <LogicalOp._or: 'or'>, <LogicalOp._not: 'not'>])\nop\n  value is not a valid enumeration member; permitted: '=', '<>', '<', '<=', '>', '>=', 'isNull' (type=type_error.enum; enum_values=[<ComparisonOp.eq: '='>, <ComparisonOp.neq: '<>'>, <ComparisonOp.lt: '<'>, <ComparisonOp.lte: '<='>, <ComparisonOp.gt: '>'>, <ComparisonOp.gte: '>='>, <ComparisonOp.is_null: 'isNull'>])\nop\n  value is not a valid enumeration member; permitted: 's_intersects' (type=type_error.enum; enum_values=[<SpatialIntersectsOp.s_intersects: 's_intersects'>])\nargs -> 1 -> op\n  field required (type=value_error.missing)\nargs -> 1 -> args\n  field required (type=value_error.missing)\nargs -> 1 -> property\n  field required (type=value_error.missing)\nargs -> 1 -> timestamp\n  field required (type=value_error.missing)\nargs -> 1 -> date\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> geometries\n  field required (type=value_error.missing)\nargs -> 1\n  int() argument must be a string, a bytes-like object or a real number, not 'dict' (type=type_error)\nargs -> 1\n  str type expected (type=type_error.str)\nargs -> 1\n  value could not be parsed to a boolean (type=type_error.bool)"
}

To Reproduce Run the temporal intersection query examples listed in the filter API extension documentation: https://github.com/stac-api-extensions/filter/?tab=readme-ov-file#example-6-temporal-intersection

Expected behavior These queries should work for full support of the filter extension.

[dipstef78]

I am facing the exact same problem. Is anybody trying to find a solution?

Thank you.

[jamesfisher-geo]

The implementation of the Filter extension here does not yet include temporal filters.

You can use the datetime field alongside filter for temporal search.

{
"datetime": ["2020-11-11T00:00:00Z", "2020-11-12T00:00:00Z"],
"filter": {}
}

Would this work?

[sotosoul]

I noticed the NoneType: None, which is something I'm also getting. Seems to me that it has to do with the chosen logging level (logging.exception vs .error). I'd like it to go away -- has this been resolved elsewhere?

[jamesfisher-geo]

@sotosoul Does my solution above work for you?

The issue here is that the T_INTERSECTS search operator is not supported in the implementation here. Happy to review any PRs if you are willing to contribute that feature.