expect_column_max_to_be_between not working for Date column

[franciskuttivelil]

Hello Support,

Getting the below error. The expectation works on other tables except the below. Have checked the table and cannot find anything different from other tables. Please help resolve this issue.

The expectation checks if the data in the table is fresh. It does that by checking if the date column has yesterday's date in it. max(date) >= yesterday's date

Code:

validator.expect_column_max_to_be_between(column=date_col, min_value=(datetime.now()-timedelta(days=last_day_delta)).replace(hour=0, minute=0, second=0, microsecond=0))

Error:

expect_column_max_to_be_between raised an exception:
Metric ('column.max', 'a3473ad0fd2ad330bcda16b9b0353e40', 'parse_strings_as_datetimes=False') is not available for validation of { "expectation_type": "expect_column_max_to_be_between", "kwargs": { "column": "Date", "min_value": "2024-01-25 00:00:00", "batch_id": "ds_clientcode_clientname_google_ctv_datasource-ds_clientcode_clientname_google_ctv_data_asset" }, "meta": {} }. Please check your configuration.
Show exception traceback...

Traceback (most recent call last):
File "/usr/share/tomcat10/.local/lib/python3.11/site-packages/great_expectations/validator/validator.py", line 1074, in graph_validate
result = configuration.metrics_validate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/tomcat10/.local/lib/python3.11/site-packages/great_expectations/core/expectation_configuration.py", line 1494, in metrics_validate
return expectation_impl(self).metrics_validate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/tomcat10/.local/lib/python3.11/site-packages/great_expectations/expectations/expectation.py", line 1061, in metrics_validate
_validate_dependencies_against_available_metrics(
File "/usr/share/tomcat10/.local/lib/python3.11/site-packages/great_expectations/expectations/expectation.py", line 3739, in _validate_dependencies_against_available_metrics
raise InvalidExpectationConfigurationError(
great_expectations.exceptions.exceptions.InvalidExpectationConfigurationError: Metric ('column.max', 'a3473ad0fd2ad330bcda16b9b0353e40', 'parse_strings_as_datetimes=False') is not available for validation of {
"expectation_type": "expect_column_max_to_be_between",
"kwargs": {
"column": "Date",
"min_value": "2024-01-25 00:00:00",
"batch_id": "ds_clientcode_clientname_google_ctv_datasource-ds_clientcode_clientname_google_ctv_data_asset"
},
"meta": {}
}. Please check your configuration.

Thanks

Environment :

• Operating System: Linux
• Great Expectations Version: 0.18.7
• Data Source: Pandas
• Cloud environment: AWS

[franciskuttivelil]

Did anyone get a chance to look at this issue?

[molliemarie]

Hello @franciskuttivelil. With the launch of Great Expectations Core (GX 1.0), we are closing old issues posted regarding previous versions. Moving forward, we will focus our resources on supporting and improving GX Core (version 1.0 and beyond). If you find that an issue you previously reported still exists in GX Core, we encourage you to resubmit it against the new version. With more resources dedicated to community support, we aim to tackle new issues swiftly. For specific details on what is GX-supported vs community-supported, you can reference our integration and support policy.

To get started on your transition to GX Core, check out the GX Core quickstart (click “Full example code” tab to see a code example).

You can also join our upcoming community meeting on August 28th at 9am PT (noon ET / 4pm UTC) for a comprehensive rundown of everything GX Core, plus Q&A as time permits. Go to https://greatexpectations.io/meetup and click “follow calendar” to follow the GX community calendar.

Thank you for being part of the GX community and thank you for submitting this issue. We're excited about this new chapter and look forward to your feedback on GX Core. 🤗