search

SSPkrolik / nimongo

Pure Nim lang MongoDB driver
http://sspkrolik.github.io/nimongo
MIT License
101 stars 20 forks source link

can't connect to mongodb atlas (mongodb hosted on aws, with `mongodb+srv://` uri) #87

Open timotheecour opened 3 years ago

timotheecour commented 3 years ago

@SSPkrolik

I can't connect to mongodb atlas (mongodb hosted on aws)

Steps to reproduce the behavior:

  1. create a mongodb instance on https://www.mongodb.com/cloud/atlas/signup (free tier available)
  2. try to connect from nimongo: 
    let uri = "mongodb+srv://myuser:mypassword@mongoatlasxxx.yyy.mongodb.net/sample_airbnb?retryWrites=true&w=majority"
# connection string provided in "Add your connection string into your application code"
let db = newMongoDatabase(uri)

    returns nil, and on inspection (modifying proc connect*(m: Mongo): bool = to avoid try/except), it fails in:

    /Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(963) newMongoDatabase
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(955) newMongoDatabase
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(941) connect
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/net.nim(1844) connect
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/net.nim(1809) connectAsync
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/nativesockets.nim(275) getAddrInfo
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/includes/oserr.nim(94) raiseOSError
Error: unhandled exception: Additional info: "nodename nor servname provided, or not known" [OSError]

m.host resolves to: mongoatlasxxx.yyy.mongodb.net so the mongodb+srv:// is lost; and I'm not sure how to instrument the code to use that ; naive attempts failed.

Expected behavior works

Desktop (please complete the following information):

note + links

JohnAD commented 3 years ago

Two challenges, at least the last time I checked: I don't believe SRV lookups work yet, so you will need to connect to a single server in the cluster.

Also, unless you have a high end account, Atlas forces you to SCRAM256 auth (rather than standard SCRAM1). That is a quasi unique change to force smaller users to Compass, their commercial client.

Any hosting provider other than Atlas will give you SCRAM1. Myself, I switched to ScaleGrid.

But, my knowledge is six months old, so take it with a grain of salt.

On a different project, I've been experimenting with cluster support (at least with explicit references.)

Your question makes me wonder if someone has solved SRV in Nim. I might ask the forum tonight.

timotheecour commented 3 years ago

thanks for your quick reply! I've looked at https://www.mongodb.com/blog/post/mongodb-3-6-here-to-SRV-you-with-easier-replica-set-connections?jmp=fcb&utm_source=4244&utm_medium=FBPAGE&utm_term=4&linkId=50841309 and ran the python code mentioned in We can see how this works in practice on a MongoDB Atlas cluster with a simple Python script. and extracted the 1st entry, and than verified it works on mongo shell via:

$homebrew_D/opt/mongodb-community-shell/bin/mongo "mongodb://myuser:mypassword@mongoatlas1-shard-00-00.xxx.mongodb.net:27017/test?ssl=true&authSource=admin&replicaSet=atlas-yyy-shard-0"

but then using that uri fails with nimongo with:

/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(969) newMongoDatabase
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(965) newMongoDatabase
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(848) authenticateScramSha1
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(404) one
/Users/timothee/git_clone/nim/nimongo/nimongo/mongo.nim(310) performFind
/Users/timothee/.choosenim/toolchains/nim-1.2.0/lib/pure/streams.nim(570) readInt32
/Users/timothee/.choosenim/toolchains/nim-1.2.0/lib/pure/streams.nim(382) read
Error: unhandled exception: cannot read from stream [IOError]

so I guess this is because of your point regarding SCRAM256 vs SCRAM1 ?

just trying to get anything working, happy to use whatever temporary workarounds; any idea of whether I could be using a C/C++ library and wrap that for the authenticateScramSha1 part?

Also, unless you have a high end account, Atlas forces you to SCRAM256 auth (rather than standard SCRAM1). That is a quasi unique change to force smaller users to Compass, their commercial client.

I just tried on M10 (non free tier), I don't see an option for SCRAM1 ? maybe I'm missing something?

links

EDIT: see https://github.com/ba0f3/scram.nim which mentions: Salted Challenge Response Authentication Mechanism (SCRAM-SHA-1 SCRAM-SHA-256 SCRAM-SHA-512)

JohnAD commented 3 years ago

Wow, @ba0f3 is supporting SCRAM-SHA-256 now. Thanks for the update on that. I'll have to add SCRAM-256 to my fork at mongopool.

The error you are seeing looks very familiar as my fork was giving the same errors. Apparently as Nim (the language) upgraded to newer versions, the internals of ba0f3's SHA and SCRAM libraries also had to change. ba0f3 was very useful in helping me fix my fork (mine is for pooled connections to the connections). If I get time this weekend, I'll see if I can fix nimongo also.

JohnAD commented 3 years ago

As to Atlas, I just jumped to my account also. I don't see it either. It used to be a user setting under "Database Access" for the cluster. But it appears to be unchangeable now. There might be some secret setting somewhere that Mongodb.com tech support can find.

JohnAD commented 3 years ago

(BTW, I created the fork as I was wanting to use it on a web server and adding pooling fundamentally changed the nature of the library IMO.) Nimongo is very fast, faster than mongopool. But mongopool safely lives inside Jester threads and shares an existing preconnected pool. This is important since SCRAM1 can take 3 or 4 seconds to authenticate. I learned this the hard way when I discovered it took 3 seconds to bring up a web page 😃 without pooling.

ba0f3 commented 3 years ago

I will add SRV support to my dnsclient package

timotheecour commented 3 years ago

the authentication logic is very complex (and many things are not supported in nimongo, eg see authentication options in mongo-c-driver/src/libmongoc/doc/authentication.rst, eg the newer MONGODB-AWS since mongodb 4.4).

Wouldn't it be much simpler if nimongo would wrap libmongoc instead of re-inventing the wheel here, at least for things like authentication?

JohnAD commented 3 years ago

For most programming languages, the available libraries for MongoDB are simply thin wrappers of the C libmongoc library. (In fact, other than Nim, the only exceptions I know of are Swift and Rust.)

But yes, a wrapper of libmongoc would work quite well. When I forked nimongo, I also spun off the BSON into a standalone library and made sure it was binary-compatible; so half the work is done by including that. https://github.com/JohnAD/bson

(off-topic: This reminds me: now that I've written a binary-compatible Decimal128 library for Nim, I need to add it to the BSON to finish out the spec. The only remaining types missing are the deprecated and almost-deprecated ones such as DBPointer.)

timotheecour commented 3 years ago

an ideal mongo library for me would wrap libmongoc for most things and provide a friendly high level wrapper around it. But https://github.com/nim-lang/mongo unfortunately is described as "unsuable" in the readme...

timotheecour commented 3 years ago

BTW, I created the fork as I was wanting to use it on a web server and adding pooling fundamentally changed the nature of the library IMO

@JohnAD does https://github.com/mashingan/anonimongo fit your requirement? eg you can do: var mongo = newMongo(parseURI uriserver, poolconn = 2) as shown here https://github.com/mashingan/anonimongo/issues/5#issuecomment-671200396

(in the spirit of having 1 feature complete library as opposed to many that all have missing key features)