Open johnwyles opened 5 years ago
Agreed! There should be an easy way to spin up a container, with a new database, username and password - all specified in docker-compose
/ environment variables.
I understand your confusion. We only create a user with the root
role (ie superuser as is done in the Postgres image) and users expect that the user is created in the database that they specified by MONGO_INITDB_DATABASE
. This is not possible for the root
role. The root
role only exists in the admin
database (probably for security reasons). Any other role not created in the admin database would not have full access to control MongoDB.
See also,
As for the database not existing, that is just how MongoDB works as I have explained before. If nothing is inserted the database does not exist (or conversely every database exists, you just have to use it).
I'm mostly against adding more environment variables for creating a non-admin user, especially if it breaks backwards compatibility. We try to keep the images as close to upstream releases as possible with minimal maintenance and the entrypoint script is already very complex.
A simple Dockerfile:
FROM mongo:4.0
COPY custom-user.sh /docker-entrypoint-initdb.d/
and a short custom-user.sh
file:
#!/bin/bash
set -e;
# a default non-root role
MONGO_NON_ROOT_ROLE="${MONGO_NON_ROOT_ROLE:-readWrite}"
if [ -n "${MONGO_NON_ROOT_USERNAME:-}" ] && [ -n "${MONGO_NON_ROOT_PASSWORD:-}" ]; then
"${mongo[@]}" "$MONGO_INITDB_DATABASE" <<-EOJS
db.createUser({
user: $(_js_escape "$MONGO_NON_ROOT_USERNAME"),
pwd: $(_js_escape "$MONGO_NON_ROOT_PASSWORD"),
roles: [ { role: $(_js_escape "$MONGO_NON_ROOT_ROLE"), db: $(_js_escape "$MONGO_INITDB_DATABASE") } ]
})
EOJS
else
# print warning or kill temporary mongo and exit non-zero
fi
Combine that with automated builds (https://docs.docker.com/docker-hub/builds/) and repository links (https://docs.docker.com/docker-hub/builds/#repository-links) and it's reasonably easy to have an up-to-date image built FROM mongo
with the custom non-root user creation modifications.
$ docker run -d -e MONGO_INITDB_ROOT_USERNAME=root -e MONGO_INITDB_ROOT_PASSWORD=supersecret -e MONGO_INITDB_DATABASE=notadmin -e MONGO_NON_ROOT_USERNAME=normal -e MONGO_NON_ROOT_PASSWORD=secret --name some-mongo custom-mongo
$ docker run -it --rm --link some-mongo mongo:4.0 mongo -u normal -p secret some-mongo/notadmin
MongoDB shell version v4.0.6
connecting to: mongodb://some-mongo:27017/notadmin?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("0941eaa4-9180-4012-a24c-60fe8369b06a") }
MongoDB server version: 4.0.6
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
> use admin;
switched to db admin
> db.users.findOne();
2019-02-05T23:57:10.838+0000 E QUERY [js] Error: error: {
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { find: \"users\", filter: {}, limit: 1.0, singleBatch: true, lsid: { id: UUID(\"c86d23b2-9ed4-4799-9d3e-74f327366550\") }, $db: \"admin\" }",
"code" : 13,
"codeName" : "Unauthorized"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DBCommandCursor@src/mongo/shell/query.js:708:1
DBQuery.prototype._exec@src/mongo/shell/query.js:113:28
DBQuery.prototype.hasNext@src/mongo/shell/query.js:288:5
DBCollection.prototype.findOne@src/mongo/shell/collection.js:260:10
@(shell):1:1
> use notadmin;
switched to db notadmin
> db.users.findOne();
null
This unclear unless you look at docker-entrypoint.sh in this repository and no where clearly stated in the documentation as such
This seems clear to me (emphasis added):
This variable allows you to specify the name of a database to be used for creation scripts in
/docker-entrypoint-initdb.d/*.js
(see Initializing a fresh instance below). MongoDB is fundamentally designed for "create on first use", so if you do not insert data with your JavaScript files, then no database is created.
@tianon has a PR for improving the user section: https://github.com/docker-library/docs/pull/1418
@yosifkit Thank you for that response. Given the majority of developers are writing their code and initialization and configuration such that they assume (whether rightly or wrongly) that their database is there and a username and password they know is able to talk to and operate on it I think it would be a great addition to the image.
Given your excellently laid out example above I think this should be baked in and not putting onus on the users of this image to essentially all have copies of the script you mentioned running around in it's various permutations. What do you say to including it in the image:
MONGO_NON_ROOT_USERNAME
and MONGO_NON_ROOT_PASSWORD
if they like and get expected behavior and if they do not then nothing happens as expected as wellI think this is excellent if you were driving at building this into the image and see no reason not to since 99% of cases users will want a pre-initialized database with a username and password - putting this in documentation will help but having the extra step of them essentially copy/pasting your script so they can get what would be expected behavior of an image with an upstream project like this where the vast majority of developers and users are not having to write the database / username/password initialization into their code after connecting with a root
role user. It also centralizes the location credentials can be kept and modified. Were it not for your example script other folks might have, understandably, done it like I did where I keep credentials in a .js
file for the user database and MONGO_INITDB_ROOT_USERNAME
and MONGO_INITDB_ROOT_PASSWORD
in docker-compose.yml
(or k8s or whatever).
And of course simply adding your custom-user.sh
code instead of in the /docker-entrypoint-initdb.d/
directory it would, of course, instead live alongside the rest in docker-entrypoint.sh
.
I understand the reluctance to add functionality to working software. I also understand @yosifkit's response, and I'm using his advice in my own code.
But this functionality is obvious and necessary - as evidenced by all those who complained about it in #174 and StackOverflow. We assumed it to work the way we've described, because that is the obvious need.
One of the reasons we use docker is the promise of "just spinning up" a new container that is ready for action. Extra steps and config (that exists in multiple places, and liable to fall out of sync) are an extra burden, which are easily avoided by the (backwards compatible) proposed additions.
I have submitted PR https://github.com/docker-library/mongo/pull/331 as outlined by @yosifkit comments and code example as well taking into account his comments:
I'm mostly against adding more environment variables for creating a non-admin user, especially if it breaks backwards compatibility. We try to keep the images as close to upstream releases as possible with minimal maintenance and the entrypoint script is already very complex.
This does not break backwards compatibility and is a fully optional feature added for expected user behavior as itterated above by @lonix1 @johnwyles in this issue and numerous others in issue https://github.com/docker-library/mongo/issues/174#issuecomment-460566429.
Combine that with automated builds (https://docs.docker.com/docker-hub/builds/) and repository links (https://docs.docker.com/docker-hub/builds/#repository-links) and it's reasonably easy to have an up-to-date image built
FROM mongo
with the custom non-root user creation modifications.
While more mature software projects will certainly require /docker-entrypoint-initdb.d/*
files the image as it is out of the box is unusable without an extra step of gaining root
access and performing the user and password creation on the DB that /docker-entrypoint-initdb.d/
operates against. In fact without a stanza in these files that creates a username and password then users would be left with the default behavior to use the root
role to perform all operations. To me this is bad security prescribed by the approach of not having a non-root username and password set. Most users at the start of their projects and when they are in less maturity and without security conscious concerns would simply use this root
user which is bad practice and an anti-pattern to be using this level of access.
And then I will just echo all the points in: https://github.com/docker-library/mongo/issues/329#issuecomment-460902952
MONGO_NON_ROOT_USERNAME
and MONGO_NON_ROOT_PASSWORD
if they like and get expected behavior and if they do not then nothing happens as expected as wellI will also put together a PR for the documentation update if we can proceed on this 👍
I have also added the documentation for this in PR https://github.com/docker-library/docs/pull/1422 in the documentation as well.
BUMP What did we think about PR https://github.com/docker-library/mongo/pull/331 and https://github.com/docker-library/docs/pull/1422 to provide users with expected behavior, backwards compatibility, and optional parameters but that when present provide the expectation the user requires for database initialization? Let me know if the PR needs a bit of modification but the discussions here seem to have all been addressed in the PR and documentation PR that provide comfortably merging this in.
Actually, I've posted a comment in a different issue, but it seems to be very mongo initialization process related, so that will leave it here also #323
https://github.com/docker-library/mongo/issues/323#issuecomment-474243238
To summarize: there are problems with MONGO_NON_ROOT_USERNAME
and MONGO_NON_ROOT_PASSWORD
when applying this variables to mongod instances in configsvr mode. Detailed description could be found by link above
@npiskunov, on your issue it looks like we just need to account for the default dbpath
change for configsvr
(#341).
On a related note, this also means that the PR for this issue(#331) has another edge case. The non-root user values cannot work when running as a config server since it cannot create a user outside of admin
(or config
).
When running with this option, clients (i.e. other cluster components) cannot write data to any database other than
config
andadmin
.- https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-mongod-configsvr
Finding more edge cases like this is why I am so hesitant to add more environment variables.
Admittedly this means that, apart from #331, we might want to note that limitation for /docker-entrypoint-initdb.d/
scripts as well as MONGO_INITDB_DATABASE
, but I haven't seen an issue about this, so it seems like a rare case.
@yosifkit Did I get right, that changes in #341 fix behavior of MONGO_NON_ROOT_USERNAME in configsvr by pointing user create on correct DB (admin)? In this case MONGO_INITDB_DATABASE
becomes useless for configsvr mode... I believe it would be useful to mention this limitation in docs and maybe even in container logic, e.g. log message "MONGO_INITDB_DATABASE is not supported in config server mode"?
Regarding #341. In my opinion changes described (mongo initialization variables) become clearer, but still not fully clear :) It would be perfect if there was e.g. compose.yml illustrating minimalistic sharded cluster composition with security (3-members RS for shard, 3-members CSRS, single mongos. Root user / Security Key file / sample DB initial seed);
Why this example is not in the documentation?
#!/bin/bash set -e; # a default non-root role MONGO_NON_ROOT_ROLE="${MONGO_NON_ROOT_ROLE:-readWrite}" if [ -n "${MONGO_NON_ROOT_USERNAME:-}" ] && [ -n "${MONGO_NON_ROOT_PASSWORD:-}" ]; then "${mongo[@]}" "$MONGO_INITDB_DATABASE" <<-EOJS db.createUser({ user: $(_js_escape "$MONGO_NON_ROOT_USERNAME"), pwd: $(_js_escape "$MONGO_NON_ROOT_PASSWORD"), roles: [ { role: $(_js_escape "$MONGO_NON_ROOT_ROLE"), db: $(_js_escape "$MONGO_INITDB_DATABASE") } ] }) EOJS else # print warning or kill temporary mongo and exit non-zero fi
to echo @carloschneider and @lonix1 - @yosifkit I think there is an "expectation" that the legwork not be on the implementer or user of this to have some expectation to add a lot of custom work and have to wade through these github PR's and Issues to find out how to implement what should otherwise be straightforward behavior one would expect - I think pushing PR https://github.com/docker-library/docs/pull/1421/files (docs) (which was closed) but also something similar or the PR I originally filed on this: https://github.com/docker-library/mongo/pull/331/files will resolve a lot of headache you clearly can see causing people to pull their hair out only to find they need to write some custom code and implementation with what should otherwise be expected or "obvious" behavior - sorry for getting back to this so late as I have been caught up in a lot of other work which let this linger for longer than it should have
This is exactly what my development environment needs!
I want to test my application's connection to a "production like" mongo instance with authentication and so on.
I'm able to do this with my local docker-compose utilizing the custom made .js
or .sh
files.
But I'm unable to do so in our CI/CD pipeline, because I can't control this behavior through environment variables.
This feature could provide a centralized way to look up your mongo credentials and your application credentials in a clear and consistent manner in your docker-compose.yml or other configuration files.
Cheers for the efforts!
Can someone post an example of how to run the docker commands without --link and using --network? I can't seem to get my nodeJS app to connect to the mongo container.
Why is a custom non root user still not a part of the default Dockerfile?
People need it and people end up implementing it themselves with custom-user.sh
!
It could save so many hours of work if the maintainer would do it once and folks can just add some environment variables instead of creating an additional dockerfile + script...
You can use the standard --user
way of running as an arbitrary user
https://github.com/docker-library/mongo/issues/315#issuecomment-439981173
$ mkdir db && sudo chown 777 db
$ docker run --rm -dit -v $PWD/db:/data/db -v /etc/passwd:/etc/passwd:ro --user 1000:1001 --name mongo mongo
6c4a4e74b314a2a01893bc0b7405106db3100db37e9ad466772f56f2f88ec2b2
$ echo $UID
1000
$ docker exec -it mongo bash
groups: cannot find name for group ID 1001
rei@6c4a4e74b314:/$ echo $UID
1000
Bumping this because I find it a little frustrating that I'm blocked because I can't initialize a mongo container with a usable database + username/password pair.
user @lonix1 stated it better than I could:
One of the reasons we use docker is the promise of "just spinning up" a new container that is ready for action. Extra steps and config (that exists in multiple places, and liable to fall out of sync) are an extra burden, which are easily avoided by the (backwards compatible) proposed additions.
My use case is simple: A MERN application where a single docker-compose.yml sets up the requisite stack ahead of my node application starting. *note: my stack doesn't include bash, it is not named BMERN or MERNB -- it's MERN.
Can I connect to my mongodb container with a username and password on application start-up and begin using my target database without issue from a single docker-compose.yml file?
No, because environment variables don't cover this extremely basic configuration.
Has this been reconsidered since 2019, @yosifkit?
UPDATE: Went with bintami/node
and never looked back. This feels like a failure to deliver on a community agreement but it's not a hill worth dying on.
Can we please get some documentation added and add the init script to run as part of the entrypoint? I've just spent the last 45 minutes digging around trying to understand why my container was not creating my databases on startup.
@CodePint That is the point of the ticket so not sure if you were just echoing it, but you can see for help: Docs: https://github.com/docker-library/docs/pull/1422 Initialization: https://github.com/docker-library/mongo/pull/331
Other than getting the maintainers to merge those tickets you are going to be stuck where a lot of folks are on these two open PRs.
I added the following to my init.sh
script that basically runs the whole app
# ! update mongodb user for authentication
sleep 10
export $(sed 's/[[:blank:]]//g; /^#/d' .env | xargs)
docker exec -it mongodb bash -c "mongo $MONGO_DB \
-u $MONGO_USERNAME \
-p $MONGO_PASSWORD \
--authenticationDatabase admin \
--eval 'db.createUser({user: \"$MONGO_USERNAME\", pwd: \"$MONGO_PASSWORD\", roles:[{role:\"userAdminAnyDatabase\", db: \"admin\"}]});
db.grantRolesToUser(\"$MONGO_USERNAME\",[{ role: \"root\", db: \"admin\" }]);'"
I'd like to leave it here in case some poor soul is in need of it
it was my first time using mongodb the other day and I went for docker straight away(big mistake). I spent the better part of 2 days trying to figure out a way around all this that didn't involve volume mounting or overriding the entry_point.sh file
it's not the cleanest solution but it does the job for now
Thanks everyone for your effort and documentation. I agree that it would be helpful to have an easy way to use docker to set up a local connection with default values without adding bash script files.
After spending all evening last night and this morning trying to figure out why these environment variables "were not working", I finally found this discussion. I would like to add my support for this feature. I'm considering moving away from this image and using Bitnami's image instead for this very reason.
That is the point of the ticket so not sure if you were just echoing it, but you can see for help: Docs: docker-library/docs#1422 Initialization: #331
Other than getting the maintainers to merge those tickets you are going to be stuck where a lot of folks are on these two open PRs.
@johnwyles
I was just just echoing and voicing my shared frustration.
Thanks for your work with these PR's, its a shame there is no movement towards getting them merged.
After spending all evening last night and this morning trying to figure out why these environment variables "were not working", I finally found this discussion. I would like to add my support for this feature.
@draeder Thanks for mentioning Bitnami's image where this issue has been solved and the solution clearly documented. This looks like a viable solution for those that are frustrated with the lack of movement and rigidity from the maintainers here.
I will be using their image in the future. As an aid to frustrated developers who will visit this page over the coming years, I've included the relevant info for the Bitnami MongoDB image below:
GitHub Repository: https://github.com/bitnami/bitnami-docker-mongodb
Running the docker image with "a user defined database, username, and password using environment variables":
# Creating a user and database on first run
# You can create a user with restricted access to a database while starting the container for the first time.
# To do this, provide the MONGODB_USERNAME, MONGODB_PASSWORD and MONGODB_DATABASE environment variables.
$ docker run --name mongodb \
-e MONGODB_USERNAME=my_user \
-e MONGODB_PASSWORD=password123 \
-e MONGODB_DATABASE=my_database bitnami/mongodb:latest
# or by modifying the docker-compose.yml file present in this repository:
services:
mongodb:
...
environment:
- MONGODB_USERNAME=my_user
- MONGODB_PASSWORD=password123
- MONGODB_DATABASE=my_database
...
For everyone running into issues using the script by @carloschneider: The script is not copy/paste, as a noop is needed in the else (at least on my machine):
#!/bin/bash
set -e;
# a default non-root role
MONGO_NON_ROOT_ROLE="${MONGO_NON_ROOT_ROLE:-readWrite}"
if [ -n "${MONGO_NON_ROOT_USERNAME:-}" ] && [ -n "${MONGO_NON_ROOT_PASSWORD:-}" ]; then
"${mongo[@]}" "$MONGO_INITDB_DATABASE" <<-EOJS
db.createUser({
user: $(_js_escape "$MONGO_NON_ROOT_USERNAME"),
pwd: $(_js_escape "$MONGO_NON_ROOT_PASSWORD"),
roles: [ { role: $(_js_escape "$MONGO_NON_ROOT_ROLE"), db: $(_js_escape "$MONGO_INITDB_DATABASE") } ]
})
EOJS
else
true # print warning or kill temporary mongo and exit non-zero
fi
As someone who's spent the last day or more, trying to make this container work in Azure Container Instances / App Services / Container Apps, I'm laughing at how ignorant I am and how bad the documentation is.
I've inserted the required MONGO_INITDB_ROOT_USERNAME / MONGO_INITDB_ROOT_PASSWORD and MONGO_INITDB_DATABASE, got the container working with an Azure File Share and then spent hours debugging why the user and database hasn't been creating.
All of the files are present in the Azure File Share though, indicating that persistent storage will work.
Thank you to everyone for providing the missing steps. It seems there's no real way to have this container build out of terraform with the required .js files to actually create the user/db. It's a shame the scripts aren't included by default.
It's been more than 3 years and it's absolutely funny how such a simple thing is so misleading for many developers wanting to quickly spin up a db instance. The frustration is more about a simple set of environment variables not working as expected, and developers wasting a ton of time in diagnosing why the connection isn't going through.
It's been more than 3 years and it's absolutely funny how such a simple thing is so misleading for many developers wanting to quickly spin up a db instance. The frustration is more about a simple set of environment variables not working as expected, and developers wasting a ton of time in diagnosing why the connection isn't going through.
correct, just killed 3 hours until figure it out, then went here to write a comment to support this feature (bug), so that in next 3 year someone fix it, and my children would not be as angry as I am right now.
Please add this, it would be so much easier and logical to have a way to create the image with a default DB, User, Password. It's just adding unecessary work, that almost everyone will do, or else they will just have a poorly secured DB or use something else.
Thanks for mentionning bitnami image in the answers above!!
How much time and money could be saved with this issue and script are included in the docs or variables name be cleaner and logical.
This issue (I would call it a bug but perhaps it is a feature request) is that users would like to a la
docker-compose.yml
and/or environment variables be able to set a database with a username and password they specify upon launch of the image.Background:
This issue was filed https://github.com/docker-library/mongo/issues/174 and closed because the behavior of a PR https://github.com/docker-library/mongo/pull/145 was mentioned as the solution. What https://github.com/docker-library/mongo/pull/145 actually does and what users expect are entirely different. What PR https://github.com/docker-library/mongo/pull/145 does is set a user with elevated permissions (i.e. "root" user) that has superuser access to the entire MongoDB instance (as mentioned in https://github.com/docker-library/mongo/issues/174#issuecomment-460448991. However what most users expect from these environment variables is that a database they specify is initialized with the username and password they have set. It is confusing that these environment variables (
MONGO_INITDB_DATABASE
,MONGO_INITDB_ROOT_PASSWORD
andMONGO_INITDB_ROOT_USERNAME
) pertain to only setting a user with the roleroot
on the databaseadmin
and initializing an user specified database for.js
and.sh
scripts in/docker-entrypoint-initdb.d/
to be run against.Proposal:
We should make the environment variables very explicitly named in what they do in addition to adding others for the behavior I think most users come to expect when reading the variable names. Since it is the case most users would like their instance initialized with a database of their specification we should add this feature to meet that expectation.
MONGO_INITDB_ROOT_USERNAME
andMONGO_INITDB_ROOT_PASSWORD
MONGO_INITDB_DATABASE
as it is misleadingMONGO_INITDB_ROOT_DATABASE
and allow it to override the hardcodedadmin
databaseMONGO_USERDB_ADMIN_USERNAME
,MONGO_USERDB_ADMIN_PASSWORD
, andMONGO_USERDB_ADMIN_DATABASE
MONGO_INITDB_ROOT_USERNAME
,MONGO_INITDB_ROOT_PASSWORD
, andMONGO_INITDB_ROOT_DATABASE
will be used for theroot
role to MongoMONGO_USERDB_ADMIN_USERNAME
,MONGO_USERDB_ADMIN_PASSWORD
, andMONGO_USERDB_ADMIN_DATABASE
will be used to initialize a user specified database.js
and.sh
scripts a user supplies in/docker-entrypoint-initdb.d/
will be executed againstMONGO_USERDB_ADMIN_DATABASE
Reasons for change:
MONGO_INITDB_ROOT_PASSWORD
andMONGO_INITDB_ROOT_USERNAME
are only used for theadmin
databaseMONGO_INITDB_DATABASE
does is have operations used against it whenever a user has dropped in.js
or.sh
scripts into/docker-entrypoint-initdb.d/
. This unclear unless you look atdocker-entrypoint.sh
in this repository and no where clearly stated in the documentation as such/docker-entrypoint-initdb.d/
which then places burden on the user to maintain theroot
role credentials in environment variables which live separately from a custom.js
or.sh
script which they have to volume into the imageReferences:
Involved Persons:
@mmi-rperez @tianon @vutran1710 @yosifkit @lonix1 @johnwyles