search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.83k stars 4.06k forks source link

Destination Postgres: connection failing after SSL requirement #17989

Closed valleybay closed 1 month ago

valleybay commented 1 year ago
## Environment - **Airbyte version**: Airbyte Cloud - **OS Version / Instance**: N/A - **Deployment**: N/A - **Source Connector and version**: Not applicable - **Destination Connector and version**: 0.3.26 - **Step where error happened**: Testing connection to destination ## Current Behavior image ## Expected Behavior Destination test should pass. ## Logs State code: 08001; Message: The connection attempt failed. ## Steps to Reproduce 1. Set up Postgres destination 2. Add verify-ca requirement 3. Connection test fails ## Are you willing to submit a PR? No experience solving these kinds of issues, but would be open to try.
sh4sh commented 1 year ago

Hey there, are you able to connect to this database by other methods? Can you confirm that you have a hostssl entry for the user with clientcert=verify-ca authentication option in your pg_hba.conf?

valleybay commented 1 year ago

Hi @sh4sh! I have tested connecting to this DB with verify-ca using DataGrip, also verified just now.

sh4sh commented 1 year ago

@valleybay thanks for checking the db configuration! Can you try pinging the db from the airbyte instance to check whether it is a networking connectivity issue?

valleybay commented 1 year ago

@sh4sh we're actually running on your cloud so I'm not able to ping. I can however say that we've already had the source/destination work before (with sync working perfectly fine over a number of weeks). It stopped working when cloud changed from allowing Postgres prefer to requiring verify-ca.

sh4sh commented 1 year ago

Thanks for reporting this and running through some troubleshooting @valleybay! I've tracked down an internal ticket in the team's backlog and have ensured this report is linked to it. If the behaviour changes or you have any other updates please let us know.

valleybay commented 1 year ago

Hi @sh4sh! Is there any way for us to find some updates on this on GH or other places? This is a critical feature for us to continue using AirByte cloud. The alternative would be hosting it ourselves, I think.

sh4sh commented 1 year ago

Hey @valleybay, currently it appears to be in backlog but I'll check and see if there are any further updates. In general contacting Cloud Support will probably be your best option as they originally reported the issue.

grishick commented 1 year ago

Note: w/o any more information the only thing we can do is verify that verify-ca mode is working

marcosboscariol commented 11 months ago

Hi! Let me try to help, I was having the same issue and this video helped:

https://www.youtube.com/watch?v=gNebJ6c59JY

I start a docker with a postgres image and it worked.

evantahler commented 1 month ago

Closing - this destination has been updated significantly since this issue was posted.