search

makeusabrew / decking

A Docker helper to create, manage and run clusters of containers
http://decking.io
443 stars 36 forks source link

1.0.0 notes #58

Open makeusabrew opened 10 years ago

makeusabrew commented 10 years ago

I made some notes on a train the other day - I had no internet access so there is some overlap with issues we've already noted, and they're pretty haphazard, but still wanted to type them up. While there's no obligation at all, I wondered if any contributors (particularly @stephenmelrose) wanted to add any important features or disagreed with anything in the list. There are some questions which I'll remove when I create a proper task list (as per e.g. https://github.com/makeusabrew/bootbox/issues/220) but thought they were worth keeping for now as discussion points.

I'd like to get 1.0.0 out by the end of the month ideally (I'm away next week). Nothing in the list is too daunting so just need to carve out some time. Slipping to October is no big deal; just nice to set some targets (for myself more than anything).

First class data containers (see #56)

Do data containers ever need running or just creating? If they don't need to be run, how do they remount a host volume (e.g. when -v has /foo:/bar as opposed to just -v /foo). Do people ever use a data container for more than just data? Would we need to explicitly mark data containers as such, and if so would an attribute like "data": true against the container(s) suffice?

decking logs (see #9, sort of)

Need to create a command like decking logs [-f] [-n X] to accompany (or really, to improve on) decking attach. When this feature was first built you couldn't choose to follow logs so attach was the only option available. Now in practice attach is a bit useless as you rarely want all output since container creation.

decking attach

Reattach logic is poor and gives up after something like two minutes. Just attempt it forever but provide exponential back off. Do we need to even log the warnings when a container goes away and/or comes back? Perhaps as a new switch e.g. -q or --quiet?

decking create (see #31)

Decking create should use the API as opposed to shelling out to docker run <args>. If it did we could create every container asynchronously without having to make sure dependencies are a) created and b) running when creating a dependent container. This would also mean we could ditch running in case of dependents which has always been pretty amateur.

Also, be good to be a bit more granular with the creation status if we can, sometimes downloading an image takes ages so it looks like it's hanging on creating when in fact it's in the middle of a docker pull.

decking start

Should create missing containers after warning that it's going to do so. It's annoying particularly when messing about with definitions and manually docker rming a specific container to have to then run decking create again just to create one missing container. This would allow users to just run decking start rather than decking create then decking start when creating a new cluster.

Detect deltas on start or restart (#40). Big task but be good to do, or try. Will need some configuration to avoid always prompting. Current thinking is to add a persistent boolean to container definition (#5), over-ridable per group. Not backwards compatible but better? This flag would also come in handy for...

decking destroy (#4)

Stop and remove all containers in a cluster. Would ignore persistent and data containers. Add flags like --include-persistent to nuke persistent containers too, and possibly a separate one for dta containers (and a third one to nuke both).

decking status

Can we tabularise? Improve output for data containers. Note which containers are persistent. Split columns into ip, ports, etc. Can we show dependencies here? Quite hard to scan at the moment sometimes.

decking recreate

Wrapper for destroy then create.

Miscellaneous

stephenmelrose commented 10 years ago

Good work. Maybe add a summarised list at the top using a task list so we can check off progress?

I'll what I can do about helping out. This new contract is talking up a lot of time at the moment. Will hopefully die down in the next week or two.

makeusabrew commented 10 years ago

Aye, just wanted to dump the notes warts & all first. Hoping to make a very small start tonight.

randywallace commented 10 years ago

What I tend to do with data containers is create volumes based on ENV metadata that create one concern for data used by other containers. For example, configs. There is a startup that runs just to swap out configs based on the variable, then the container exits. This allows me to have one container that maintains all the different configs for QA, Production, etc.. etc.. configurable by a ENV variable, and usable by as many containers in an environment (er, group) that I need.

I also use another data container to pull the latest git code based on other metadata depended upon by a group. If for one group I intend to run tag v.10, then that data container when started will pull down that code, then exit.

As such, considering that these data containers do something when they run (besides just host static data) I think it would be nice to be able to say that those containers must exit before starting dependent containers. Then I don't have to put in sleep(s) in dependent containers to ensure that the data containers finished what they were doing :)

stephenmelrose commented 10 years ago

@randywallace WRT your data container pulling down the latest git code for your other containers to use, I actually think this kind of defeats the purpose of Docker. The whole point is you build once, and ship everywhere. If you're code is downloaded everytime you "ship", i.e. start a container from the "shipped" image", then you could be in a position where you have different code in different environments.

In my opinion what you should be doing is "building" your data container, tagging it as an image, and then using that image everywhere. That way you have a consistent image, and by extension codebase, regardless where or how you start it.

randywallace commented 10 years ago

@stephenmelrose I understand and acknowledge your opinion, and for my particular use-case, what I do works very well and satisfies a critical need I have to create one-off QA environments quickly and reliably from ENV Variables. If I could create a build from an ENV variable, or group of them, without having to set/define them prior to the git statements that generate the base image, I would; but, right now, I have to generate or edit a Dockerfile in order to change the build, and then reclone entire repositories during the build, which is unnecessary, time consuming, and hammers github needlessly. Performing a merge based on a desired tag or branch is quick when the base image is generally up to date and the step is performed when the data container is started, not when it is built, so I can get on with running containers from that volume. We move fast, and in any given day I could be spooling up 1 or 20 unique QA environments from topic branches, tags, Pull Requests, what have you.

I maintain the uniqueness of the aforementioned git repo container via decking.io's concept of clusters and groups via environment variables, which is why I use it on top of docker. One cluster has one group, which allows me to define those ENV variables I need for a build to be unique, repeatable, maintainable, and DRY. If my use case wasn't so darn complicated, with configurations(volumes), fixtures(volumes), dependencies(links), source(volumes), I wouldn't be using decking.io in the first place.

makeusabrew commented 9 years ago

Locking this as it's old and redundant but keeping it open because there's a lot to be said for @randywallace's contribution and I'd lose sight of it if I closed it :)