toluaina
commented
1 year ago Hi and sorry for the delay.
I have used PGSync with Elastic cloud in the past yes.
I think this depends on how Elastic cloud is authenticating.
Can you please try the latest main branch and specify ELASTICSEARCH_HTTP_AUTH
e.g ELASTICSEARCH_HTTP_AUTH=admin,admin
Normally the cloud_id would be sufficient
PGSync version: 2.3.3
Postgres version: 14
Elasticsearch version: 8.5.1
Redis version: 4.3.4
Python version: 3.10
Problem Description: Hello.
I am having issues with my connections to ES. I have an Elastic Cloud (not self hosted) and I am trying to use pgsync to sync a postgresql to our elastic cloud index. I fail to establish a connection with the Elastic cloud. I have tried to supply the following .env variables without any success.
SCHEMA='schema.json' USE_ASYNC=True ELASTICSEARCH_HOST= "......es.io" ELASTICSEARCH_PORT="XXXX" ELASTICSEARCH_SCHEME="https" ELASTICSEARCH_USERNAME=""
ELASTICSEARCH_PASSWORD=""
PG_HOST=
PG_USER=
PG_PORT=
PG_PASSWORD=
I get the following error message: elasticsearch.exceptions.AuthenticationException: AuthenticationException(401, 'security_exception', 'missing authentication credentials for REST request [/]')
I have also tried the method of using the CLOUD_ID instead of elastic host/port/scheme. That yields the following error:
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f6290893190>, 'Connection to..aws.cloud.es.io timed out. (connect timeout=10.0)')
I've tried more combinations of credentials but all fail. What am I missing? I've also tried to add the certificate without success.
Stucture of project currently: ---schema.json ---.env
Edit, Questions: -- Is pgsync compatible with elastic cloud? -- Does one have to use Redis? I am currently not using it. But do I need to set a db up in Redis in order for the pipeline of pgsync to work?