Open VolkovGeoPhy opened 3 weeks ago
Hi there! Redshift isn’t a fully supported or tested data source in Great Expectations. This means that while it can work with SQLAlchemy, certain features or expectations, like ColumnMapExpectation
, might run into issues if Redshift handles SQL slightly differently than other databases.
We generate SQL queries to validate data against your expectations. If Redshift doesn’t fully support the type of SQL query being generated, that can cause the validation to fail. Redshift may not support certain functions or data operations that the expectation relies on.
I’d recommend reviewing the SQL queries generated by the expectation to ensure they’re compatible with Redshift’s SQL dialect. You can enable SQL query logging in SQLAlchemy to see what’s being executed and compare it with Redshift’s SQL requirements. If there’s a mismatch, it may be necessary to adjust the_validate()
method in your custom expectation to work better with Redshift.
Unfortunately, as Redshift isn't supported, I will be closing this issue. But I do encourage you to call on our community for assistance if the above tip I shared doesn't get you in the right direction. Thank you for reaching out!
@adeola-ak nice but it doesn't relate to Redshift dialect. https://greatexpectationstalk.slack.com/archives/CUTCNHN82/p1729850880800489?thread_ts=1729691473.828959&cid=CUTCNHN82 It just inconsistency in your code. Check the classes that I meantioned in the description!
You have one class that inherits another. And in your child class you use validation that fails because of the structure of the parent class
@adeola-ak the same with 1.2.2 version
Describe the bug For instance I use this type of expectations. (or many other types)
class ExpectColumnValuesToMatchRegex(ColumnMapExpectation):
`class ColumnMapExpectation(BatchExpectation, ABC): """Base class for ColumnMapExpectations.To Reproduce Add expectation ExpectColumnValuesToMatchRegex
Expected behavior Empty logs
Environment (please complete the following information):
Additional context Add any other context about the problem here.