great-expectations / great_expectations

Always know what to expect from your data.
https://docs.greatexpectations.io/
Apache License 2.0
9.71k stars 1.5k forks source link

`TableExpectation.validate_metric_value_between_configuration` throws exception when comparting dicts #5221

Open cbuffett opened 2 years ago

cbuffett commented 2 years ago

Describe the bug The static method validate_metric_value_between_configuration has checks in place to validate that if the passed in min/max values are dictionaries, that they contain the $PARAMETER keyword, but then immediately attempts to compare said dictionaries using the > inequality operator, resulting in an exception:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [15], in <module>
     98 print(eval_param_urn_min)
     99 print(eval_param_urn_max)
--> 100 validator_target.expect_column_min_to_be_between(col_name, min_value={'$PARAMETER': eval_param_urn_min}, max_value={'$PARAMETER': eval_param_urn_min})
    101 validator_target.expect_column_max_to_be_between(col_name, min_value={'$PARAMETER': eval_param_urn_max}, max_value={'$PARAMETER': eval_param_urn_max})

File ~/.virtualenvs/great_expectations/lib/python3.8/site-packages/great_expectations/validator/validator.py:383, in Validator.validate_expectation.<locals>.inst_expectation(*args, **kwargs)
    377         validation_result = ExpectationValidationResult(
    378             success=False,
    379             exception_info=exception_info,
    380             expectation_config=configuration,
    381         )
    382     else:
--> 383         raise err
    384 return validation_result

File ~/.virtualenvs/great_expectations/lib/python3.8/site-packages/great_expectations/validator/validator.py:334, in Validator.validate_expectation.<locals>.inst_expectation(*args, **kwargs)
    327     configuration.process_evaluation_parameters(
    328         self._expectation_suite.evaluation_parameters,
    329         True,
    330         self._data_context,
    331     )
    333 try:
--> 334     expectation = expectation_impl(configuration)
    335     """Given an implementation and a configuration for any Expectation, returns its validation result"""
    337     if not self.interactive_evaluation and not self._active_validation:

File ~/.virtualenvs/great_expectations/lib/python3.8/site-packages/great_expectations/expectations/expectation.py:172, in Expectation.__init__(self, configuration)
    168 def __init__(
    169     self, configuration: Optional[ExpectationConfiguration] = None
    170 ) -> None:
    171     if configuration is not None:
--> 172         self.validate_configuration(configuration)
    173     self._configuration = configuration

File ~/.virtualenvs/great_expectations/lib/python3.8/site-packages/great_expectations/expectations/core/expect_column_min_to_be_between.py:213, in ExpectColumnMinToBeBetween.validate_configuration(self, configuration)
    202 """
    203 Validates that a configuration has been set, and sets a configuration if it has yet to be set. Ensures that
    204 necessary configuration arguments have been provided for the validation of the expectation.
   (...)
    210     None. Raises InvalidExpectationConfigurationError if the config is not validated successfully
    211 """
    212 super().validate_configuration(configuration=configuration)
--> 213 self.validate_metric_value_between_configuration(configuration=configuration)

File ~/.virtualenvs/great_expectations/lib/python3.8/site-packages/great_expectations/expectations/expectation.py:1613, in TableExpectation.validate_metric_value_between_configuration(configuration)
   1610 except AssertionError as e:
   1611     raise InvalidExpectationConfigurationError(str(e))
-> 1613 if min_val is not None and max_val is not None and min_val > max_val:
   1614     raise InvalidExpectationConfigurationError(
   1615         "Minimum Threshold cannot be larger than Maximum Threshold"
   1616     )
   1618 return True

TypeError: '>' not supported between instances of 'dict' and 'dict'

To Reproduce Steps to reproduce the behavior: Attempt to use the expect_column_min_to_be_between expectation with evaluation parameters

Expected behavior Evaluation parameters should be substituted prior to inequality check or dictionaries should be handled separately.

Environment (please complete the following information):

Additional context In this case, interactive evaluation was disabled.

rdodev commented 2 years ago

Hey @cbuffett Thanks for issue report. I'll be asking the dev team about this. A question though:

cbuffett commented 2 years ago

Correct. It was disabled using validator_target.interactive_evaluation = False. This is necessary, otherwise an exception is thrown because the evaluation parameters haven't actually been created in memory yet (this happens when running the checkpoint later on I believe). This is called out in Step 3 of https://docs.greatexpectations.io/docs/guides/expectations/advanced/how_to_create_expectations_that_span_multiple_batches_using_evaluation_parameters/