rwynn / monstache

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
https://rwynn.github.io/monstache-site/
MIT License
1.28k stars 180 forks source link

I can't connect to mongo atlas anymore #187

Closed Valterrsj closed 5 years ago

Valterrsj commented 5 years ago

Hi Rwynn,

I have a dockercompose that starts server elasticsearch, kibana and monstache. The monstache then synchronizes the local elasticsearch to mongoAtlas. Synchronization was normally happening after following the instructions:

However, currently with the same configuration, I am not able to connect to mongodb atlas. I get the message: Unable to connect to MongoDB using URL mongodb: //REDACTED@cluster0-shard-00-00-3huyc.mongodb.net: 27017, cluster0-shard-00-01-3huyc.mongodb.net: 27017, cluster0 -shard-00-02-3huyc.mongodb.net:27017/test?replicaSet=Cluster0-shard-0&authSource=admin&authMechanism=SCRAM-SHA-1: timed out after 15 seconds

I tried to create a new user, increase the timeout, remove and insert parameters in the URL above ... But I could not solve

rwynn commented 5 years ago

Hi @Valterrsj, I'm not sure what it could be. I'll need to try this out myself when I get some time. I have had success in the past with MongoDB Atlas but who knows what has changed. Can you check the supported auth mechanisms reported by the Atlas web UI for your user? Do you know if Atlas automatically updates the MongoDB version to the latest? Monstache only supports SCRAM-SHA-1 currently. Could it be that Atlas dropped support for this method in favor of SCRAM-SHA-256?

rwynn commented 5 years ago

The only other thing I can think of would be to check the IP whitelist to make sure you are allowed to connect from the monstache host.

Valterrsj commented 5 years ago

Thank for the quick response. Authentication mechanisms remain the same. My user is admin and by default uses scram sha 1. IP whitelist is allowed to connect from anywhere :(.

rwynn commented 5 years ago

Did the ability to connect fail after an update to a newer version of monstache? Or it just stopped working without any change to your dockercompose config?

Valterrsj commented 5 years ago

Just stopped working... I'm using this image: rwynn/monstache:4.15.0

rwynn commented 5 years ago

I tried this today and it was working OK for me.

atlas.toml

mongo-url = "mongodb://user:password@cluster0-shard-XXX.mongodb.net:27017,cluster0-shard-XXX.mongodb.net:27017,cluster0-shard-XXX.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&authMechanism=SCRAM-SHA-1"
vagrant@vagrant:~/go/bin$ docker run --rm --net=host -v /home/vagrant/atlas.toml:/tmp/atlas.toml rwynn/monstache:4.15.1 -f /tmp/atlas.toml
INFO 2019/03/15 18:41:44 Started monstache version 4.15.1
INFO 2019/03/15 18:41:44 Successfully connected to MongoDB version 4.0.6
INFO 2019/03/15 18:41:44 Successfully connected to Elasticsearch version 6.0.0
INFO 2019/03/15 18:41:44 Listening for events

IP whitelist was connect from anywhere. Cluster Version 4.0.6 on an M0. User has this auth.

Auth Method: SCRAM
MongoDB roles: read@local, readWriteAnyDatabase@admin

I also tried with change streams which worked OK

vagrant@vagrant:~/go/bin$ docker run --rm --net=host -v /home/vagrant/atlas.toml:/tmp/atlas.toml rwynn/monstache:4.15.1 -f /tmp/atlas.toml -change-stream-namespace ''
INFO 2019/03/15 18:50:07 Started monstache version 4.15.1
INFO 2019/03/15 18:50:08 Successfully connected to MongoDB version 4.0.6
INFO 2019/03/15 18:50:08 Successfully connected to Elasticsearch version 6.0.0
INFO 2019/03/15 18:50:08 Listening for events
INFO 2019/03/15 18:50:08 Watching changes on the deployment
Valterrsj commented 5 years ago

Rwynn,

The problem was in the docker internet connection. It was unstable and affecting the connection between the mongo atlas and the monstache. Using the docker-machine restart command, I solved it! Thank you very much for all support!

rwynn commented 5 years ago

Glad you got it working again.