expect_day_sum_to_be_close_to_equivalent_week_day_mean trying to find data for date '2022-08-03'

[franciskuttivelil]

Description : I am trying to use the experimental expectation expect_day_sum_to_be_close_to_equivalent_week_day_mean. It gives an exception everytime I try to run it. It seems to be using the value for TODAY that is set for tests i.e. 2022-08-03

Expectation Config :

{
      "kwargs": {
        "template_dict": {
          "summed_column": "clicks",
          "date_column": "date"
        },
        "query": "\n   \tSELECT date as date_column, SUM(clicks) as column_sum_over_date\n   \tFROM schema_name.table_name\n    \tGROUP BY date\n    \t",
        "run_date": "2023-08-16",
        "weeks_back": 1,
        "threshold": 0.25,
        "catch_exceptions": true
      },
      "expectation_type": "expect_day_sum_to_be_close_to_equivalent_week_day_mean",
      "meta": {}
    },

Implemented code :

context_ge.save_expectation_suite(suite, "{}_standard_suite".format(asset_name))
checkpoint_config = {
    "name": "{}_Checkpoint".format(table_name),
    "config_version": 1,
    "class_name":"SimpleCheckpoint",
    "expectation_suite_name": "{}_standard_suite".format(asset_name)
}
context_ge.add_checkpoint(**checkpoint_config)
results = context_ge.run_checkpoint(
    checkpoint_name="{}_Checkpoint".format(table_name),
    validations=[
        {"batch_request": batch_request}
    ]
)
#print(results)
print(context_ge.open_data_docs())

Error :

expect_day_sum_to_be_close_to_equivalent_week_day_mean raised an exception:
'2022-08-03'
[Show exception traceback...](https://mcspgra-grab-dataquality.s3.eu-west-1.amazonaws.com/data_docs/validations/apple_search_ads_spend_ds_standard_suite/__none__/20230819T073813.152224Z/309bc8d68b7033f5cbcc0793225bd9e3.html#section-2-content-block-2-cell-1-2-content-2-collapse-body-f8df2374-1cb9-4aa3-ae1d-7f50758cc1d5)

Traceback (most recent call last):
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations/validator/validator.py", line 1080, in graph_validate
result = configuration.metrics_validate(
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations/core/expectation_configuration.py", line 1486, in metrics_validate
return expectation_impl(self).metrics_validate(
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations/expectations/expectation.py", line 1065, in metrics_validate
] = self._validate(
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations_experimental/expectations/expect_day_sum_to_be_close_to_equivalent_week_day_mean.py", line 210, in _validate
diff_fraction = get_diff_fraction(yesterday_sum, result_dict, days_back_list)
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations_experimental/expectations/expect_day_sum_to_be_close_to_equivalent_week_day_mean.py", line 251, in get_diff_fraction
previous_days_sums: List[int] = [
File "/usr/share/tomcat8/.local/lib/python3.8/site-packages/great_expectations_experimental/expectations/expect_day_sum_to_be_close_to_equivalent_week_day_mean.py", line 252, in
result_dict[equiv_day] for equiv_day in equivalent_previous_days_str
KeyError: '2022-08-03'

Environment :

• Operating System: Centos
• Great Expectations Version: 0.17.5
• Data Source: Redshift
• Cloud environment: AWS

[HaebichanGX]

Hi @franciskuttivelil thank you for letting us know about this. Let me put this into backlog for our engineering review

[molliemarie]

Hello @franciskuttivelil. With the upcoming launch of Great Expectations Core (GX 1.0), we are closing old issues posted regarding previous versions. Moving forward, we will focus our resources on supporting and improving GX Core (version 1.0 and beyond). If you find that an issue you previously reported still exists in GX Core, we encourage you to resubmit it against the new version. With more resources dedicated to community support, we aim to tackle new issues swiftly. For specific details on what is GX-supported vs community-supported, you can reference our integration and support policy.

To get started on your transition to GX Core, check out the GX Core quickstart (click “Full example code” tab to see a code example).

You can also join our upcoming community meeting on August 28th at 9am PT (noon ET / 4pm UTC) for a comprehensive rundown of everything GX Core, plus Q&A as time permits. Go to https://greatexpectations.io/meetup and click “follow calendar” to follow the GX community calendar.

Thank you for being part of the GX community and thank you for submitting this issue. We're excited about this new chapter and look forward to your feedback on GX Core. 🤗