search

rwynn / monstache

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
https://rwynn.github.io/monstache-site/
MIT License
1.29k stars 181 forks source link

Authorization failure : Error on Monstache #486

Open pcinnusamy opened 3 years ago

pcinnusamy commented 3 years ago

Hi Rwynn,

I am getting below error while running Monstache and data not being synced with elastic search due to this reason , please help to review and provide your feedback on it.

INFO 2021/03/09 03:07:46 Started monstache version 6.7.4 INFO 2021/03/09 03:07:46 Go version go1.15.5 INFO 2021/03/09 03:07:46 MongoDB go driver v1.4.6 INFO 2021/03/09 03:07:46 Elasticsearch go driver 7.0.22 INFO 2021/03/09 03:07:46 Successfully connected to MongoDB version 3.6.0 INFO 2021/03/09 03:07:46 Successfully connected to Elasticsearch version 7.2.1 INFO 2021/03/09 03:07:46 Sending systemd READY=1 WARN 2021/03/09 03:07:46 Systemd notification not supported (i.e. NOTIFY_SOCKET is unset) INFO 2021/03/09 03:07:46 Listening for events INFO 2021/03/09 03:07:46 Watching changes on collection subscription.paymentschedule INFO 2021/03/09 03:07:46 Direct reads completed ERROR 2021/03/09 03:07:46 Authorization failure TRACE 2021/03/09 03:08:16 POST /_bulk HTTP/1.1 Host: es-mongoconnectors-poc:9200 User-Agent: elastic/7.0.22 (linux-amd64) Content-Length: 375 Accept: application/json Authorization: Basic *** Content-Type: application/x-ndjson

pcinnusamy commented 3 years ago

@rwynn

Could you please check and confirm what needs to be done on this issue.

rwynn commented 3 years ago

Hi @pcinnusamy when you have direct reads enabled and resume set to true then monstache attempts to read the current timestamp from MongoDB after the reads have completed. It may not have the permission it needs to do this. This is only for updating the timestamp for a subsequent run.

The relevant code is at https://github.com/rwynn/monstache/blob/d75e7854e89e87df879d717488a26ef049ec0c17/monstache.go#L4989

The queries it tries are 

db := client.Database("admin")
    result := db.RunCommand(context.Background(), bson.M{
        "replSetGetStatus": 1,
    })
db := ic.mongo.Database("admin")
    result := db.RunCommand(context.Background(), bson.M{
        "serverStatus": 1,
    })
pcinnusamy commented 3 years ago

Thanks for your detailed explanation @rwynn. So in this case we have to run Monstache service with admin or equivalent user right. Kindly suggest me which user we can use while accessing Document DB.

rwynn commented 3 years ago

You can probably check if DocumentDB supports either the replSetGetStatus or serverStatus command first. Then if at least one of those is supported then check what role the user needs to be able to call those.