Introduce volbuild, a way to bind-mount code when developing locally

[athyuttamre]

I'd like to introduce and open for discussion a new image: volbuild. This bind-mounts the codebase into the Docker container and runs meteor. An ability to provide arguments is provided.

Philosophy

I think Docker's core idea is that if something works on your laptop, it will work anywhere (including your server.) For this to be realistic, the developer must edit code while running the app within a Docker environment. However, building and running a Node bundle each time is slow, cumbersome, and non-Meteoric.

Bind-mounting code, on the other hand, allows for hot code push as well as the standard Meteor runtime, all while maintaining the isolated environment.

Some have expressed that this should've been devbuild originally. To not break backwards compatibility, I've created a new name volbuild.

Notes

If accepted, we should probably mention in the README that while by default ARGS has a value of --port 80, if it is overridden, the new value must include --port 80 for it to work as expected. Otherwise Meteor will run with port 3000. If someone has ideas to do this within the code itself (I'm new to bash scripting), by all means welcome.

Also, if you've been running your code on your host machine, sometimes the .meteor/local directory messes up builds within the container. In this case the user should delete that folder and rerun.

Finally, this method seems to show a significant delay between => Starting proxy to => Your app is running at http://domain:80. Not sure why.

[athyuttamre]

Here's a sample docker-compose.yml:

app:
  build: ./app # EXPOSEs 80
  ports:
   - 8080:80
  links:
   - db
  volumes:
   - ./app:/app # Relative to Compose file
  environment:
   - ROOT_URL=https://local.cis-dev.brown.edu
   - MONGO_URL=mongodb://db:27017/signmeup
   - VIRTUAL_HOST=local.cis-dev.brown.edu
   - ARGS=--settings ./settings.json --port 80 # Relative to project directory
   - MAIL_URL=smtp://mx.cs.brown.edu:25

db:
  image: mongo:latest
  ports:
   - 27071:27071

The developer must also include the volume mounting information, either with a Compose file, or a -v argument to docker.

[jshimko]

So forgive me if I'm missing something, but I'm not entirely clear what is gained by using this setup. This is just putting a Meteor development setup on a Debian host instead of on your local OS. I can't think of any advantages this gives you over running your local development setup like you normally would. There's certainly nothing wrong with doing this, but I guess I don't see the point of the added complexity. What does it accomplish that running on your local OS doesn't?

Also, a little side note on your docker-compose.yml example... you don't need to expose Mongo ports publicly on the host when you have your app container linked to it. The link will allow the app to have access to any ports that the database container has internally exposed without having to open the database up to the outside world. I realize this example was intended for local development, but I just wanted to point that out in case you were using a similar config in production (which would be really insecure because that Mongo image has no authentication by default and you'd be opening it up to the public internet - password free).

[athyuttamre]

Thanks for the note @jshimko!

It has two distinct advantages:

• It allows us to run the app in the same environment as production. This helps onboard developers easily (they don't have to mess with installing Meteor on their host, be it Linux, Mac, or Windows), and can rest assured that if the app runs on their computer, it won't have any environment issues on production.
• It allows us to easily use Meteor with Docker Compose. My setup is 3 containers: one for MongoDB, one for an nginx proxy, and one for the Meteor app (with volbuild). If I was running Meteor locally, I'd have to install nginx on my own computer, and run MongoDB on it too. Instead now, I can test my entire setup, all with just docker-compose up.

I think together these are compelling reasons to have such a setup for local development. The end goal is to completely make the app and its related containers independent of the host system, which we achieve here.

As for the docker-compose file, thanks for the pointer! This is only a local file, and my prod file does not have any publicly exposed ports. :)

[athyuttamre]

Relevant discussion here: https://github.com/meteorhacks/meteord/issues/57

[nunovieira]

I think devbuild should be like what's proposed here.

[athyuttamre]

Bump. @arunoda, any thoughts?

[flippyhead]

I very much like this approach and clearly see the advantages. I've got a team of developers on Linux and OSX and we've wanted a guaranteed / repeatable development environment. Vagrant is great but we'd prefer to use Docker because we also use it in production and prefer the Docker approach anyways. With the setup proposed here we can be sure new developers are onboard instantly but get the benefits of the meteor "development" runtime.

[athyuttamre]

@arunoda Pretty sure you've got your time filled, but would really appreciate your thoughts on this!

[flippyhead]

I do notice that meteor doesn't seem to run because of a missing ps command. I had to add to my Dockerfile:

RUN apt-get install procps -y

[athyuttamre]

@flippyhead Strange. Might be due to a new version of Meteor. I haven't used the image in a couple of weeks, but will check it out!

Also, I got an email about an inline comment from you, but can't find it anymore. Did you delete it?

[flippyhead]

Yes, sorry about that. Solved the issue myself.

[shaliko]

@athyuttamre Any idea about delay between => Starting proxy to => Your app is running at http://domain:80?

[athyuttamre]

@shaliko Honestly not sure. I'm not familiar with what happens in that stage of the Meteor build process, but would love to know if someone is more knowledgeable.

[shaliko]

@athyuttamre additionally I find that after change any .js file restart server require a lot of time. Is it my issue or you have the same issue?

[athyuttamre]

@shaliko Hmm, it takes me a few seconds for sure. Definitely slower than directly running Meteor on my computer. I wish I knew the details of how proxying and bind-mounting affects the build, but not an expert. :(

[shaliko]

@athyuttamre Can you share your docker-compose and .dockerignore?

My configuration https://github.com/apinf/api-umbrella-dashboard/pull/926/files#diff-4e5e90c6228fd48698d074241c2ba760R1 and restart (change any .js file) sometimes require ~1 minute :(

[athyuttamre]

@shaliko For sure, here are my files: https://github.com/signmeup/signmeup

[workflow]

@shaliko Wild guess here: I think meteor is downloading many of its modules into the .meteor/packages directory at this stage. Are you guys caching that dir somewhere?

[athyuttamre]

@workflow I believe Meteor downloads packages into .meteor/local/plugin-cache/, which should be bind-mounted along with the rest of the directory. So don't think that's the issue, but I could be wrong.

[shaliko]

@athyuttamre Thanks a lot! I install and configured your signmeup app. The same issue with long server restarting I see with your docker images. From change app/server/startup.js it takes ~30 sec for reload server.

The same for my Docker images. So at least I know that I did everything right. But continue think how speed up it.

[shaliko]

@workflow @athyuttamre Yes, Meteor packages downloads to .meteor and mounted to Docker image.