richardwilly98 / elasticsearch-river-mongodb

MongoDB River Plugin for ElasticSearch
1.12k stars 215 forks source link

connecting river to mongod with ssl .pem file? #348

Open gl00ten opened 9 years ago

gl00ten commented 9 years ago

So I created a certifcate with these instructions:

http://docs.mongodb.org/manual/tutorial/configure-ssl/

shortly:

cd /etc/ssl/
openssl req -newkey rsa:2048 -new -x509 -days 365 -nodes -out mongodb-cert.crt -keyout mongodb-cert.key
cat mongodb-cert.key mongodb-cert.crt | sudo tee mongodb.pem

I initiated my mongod proccesses

sudo mkdir -p /srv/mongodb/rs0-0 /srv/mongodb/rs0-1
sudo chmod -R 0777 /srv/
mongod --port 27000 --sslMode requireSSL --sslPEMKeyFile /etc/ssl/mongodb.pem --smallfiles --oplogSize 128 --replSet rs0 --dbpath /srv/mongodb/rs0-0
mongod --port 27001 --sslMode requireSSL --sslPEMKeyFile /etc/ssl/mongodb.pem --smallfiles --oplogSize 128 --replSet rs0 --dbpath /srv/mongodb/rs0-1

I successfully connected to them with the .pem file and

mongo --port 27000 --ssl --sslPEMKeyFile /etc/ssl/mongodb.pem

then inside mongo i initiated the replicate set

rsconf0 = {
    _id: "rs0",
    members: [
        {
            _id: 0,
            host: "ec2-54-171-150-225.eu-west-1.compute.amazonaws.com:27000"
        },
        {
            _id: 1,
            host: "ec2-54-171-150-225.eu-west-1.compute.amazonaws.com:27001"
        }
    ]
}
rs.initiate(rsconf0)

and from a shell I configured my river with this:

curl -XPUT 'https://68778b297f771269000.qbox.io/_river/inventory/_meta' -d '{
  "type": "mongodb",
  "mongodb": {
    "servers": [
        { "host": "ec2-54-171-150-225.eu-west-1.compute.amazonaws.com", "port": "27000" },
        { "host": "ec2-54-171-150-225.eu-west-1.compute.amazonaws.com", "port": "27001" }
    ],
    "options": { "ssl" : true, "sslverifycertificate" : false },
    "db": "rsTest",
    "collection": "inventory",
    "gridfs": false 
  },
  "index": {
    "name": "inventory",
    "type": "person"
  }
}'

But as expected it won't work because it doesn't have the .pem key. What do I do?

gl00ten commented 9 years ago

One thing to note, when I created the .pem file, the instructions were for a local certificate:

"For testing purposes, you can generate a self-signed certificate and private key on a Unix system with a command that resembles the following"

from here: http://docs.mongodb.org/manual/tutorial/configure-ssl/

and my elasticsearch is on another machine (hosted on qbox).

rishikeshpalve commented 9 years ago

Did you find a solution for this?

gl00ten commented 9 years ago

I think so, the solution became more apparent when we realized instead of:

mongo --host hostname --port 27000 --ssl --sslPEMKeyFile /etc/ssl/mongodb.pem

one can just use:

mongo --host hostname --port 27000 --ssl

and ssl takes care of the negotiation, I think.

Then, we just repeated the above trusting ssl to do the same thing with river, and recheck our firewall ports and it seems to be work. Will test further tomorrow.

You had any advice?

rishikeshpalve commented 9 years ago

I was just wondering if you could resolve the issue as I am planning to setup SSL for MongoDB.

When you just use --ssl option, how does the mongod instance look for .pem file? Did you add sslPEMKeyFile property in mongod.conf?

gl00ten commented 9 years ago

mongo is the shell that connects to a running mongod

Dont confuse them. :)

As you can see above, I used the .pem file to run the mongod instances. But I'm not really sure why that is necessary as then I can connect to them without referencing a .pem file.

I'm not sure how this is working in the background, but imagine that with just using the --ssl parameter to connect to mongod, the peers trade their public keys before they start talking to each other. This seems true for both mongo and elasticsearch-river-mongodb

I imagine that the .pem file is there so you can have your connection certified by a third party. But then again, that doesn't make much sense because such a file is mandatory and not optional, when starting a mongod with ssl enabled.

So to answer your questions: I'm not sure how or if mongo looks for the .pem file. I'm sure it can't do that from an external machine and it still works, so I'm guessing it doesn't actually use it. And no, I did not add a property to mongod.conf, although that may have been generated from my instantiation.

Of course this all has the fundamental problem that in this setup anyone can just mongo --ssl into my machine. <- (this issue is still open because of this)

Hope this helps you, I'm still a bit confused myself :)

linusyong commented 9 years ago

@fullmooninu What's your sslMode for mongod? requireSSL, preferSSL or allowSSL?

gl00ten commented 9 years ago

@linusyong it's up there, requireSSL