search

great-expectations / great_expectations

Always know what to expect from your data.
https://docs.greatexpectations.io/
Apache License 2.0
9.86k stars 1.52k forks source link

Replace sqlalchemy-redshift with supported library like redshift-connector #9845

Closed VolkovGeoPhy closed 1 month ago

VolkovGeoPhy commented 5 months ago

Is your feature request related to a problem? Please describe. FOr redshift right now GX uses sqlalchemy-redshift which is barely alive. This project is not developed properly. Can we use something like redshift-connector

Describe the solution you'd like I'd like to have mantainable libraries in project's dependancies

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context

Kilo59 commented 4 months ago

@VolkovGeoPhy what happens if you install this dependency locally and attempt to use it as the data-connector?

The documentation for sqlalchemy-redshift says that it can use either redshift_connector or pyscopg2.

This dialect requires either redshift_connector or psycopg2 to work properly. It does not provide it as required, but relies on you to select the distribution you need https://github.com/sqlalchemy-redshift/sqlalchemy-redshift?tab=readme-ov-file#installation

VolkovGeoPhy commented 3 months ago

Because we have this dependency GX cannot recognise NUMERIC type. Check for data type doesn't work for Redshift for this type @Kilo59

molliemarie commented 1 month ago

Hello @VolkovGeoPhy. With the launch of Great Expectations Core (GX 1.0), we are closing old issues posted regarding previous versions. Moving forward, we will focus our resources on supporting and improving GX Core (version 1.0 and beyond). If you find that an issue you previously reported still exists in GX Core, we encourage you to resubmit it against the new version. With more resources dedicated to community support, we aim to tackle new issues swiftly. For specific details on what is GX-supported vs community-supported, you can reference our integration and support policy.

To get started on your transition to GX Core, check out the GX Core quickstart (click “Full example code” tab to see a code example).

You can also join our upcoming community meeting on August 28th at 9am PT (noon ET / 4pm UTC) for a comprehensive rundown of everything GX Core, plus Q&A as time permits. Go to https://greatexpectations.io/meetup and click “follow calendar” to follow the GX community calendar.

Thank you for being part of the GX community and thank you for submitting this issue. We're excited about this new chapter and look forward to your feedback on GX Core. 🤗