docker-compose-api forgets which containers it has created between restarts

[jeroenj]

When starting a container from a ruby process a container with a random name gets created. It keeps track of the container by using it's id in-memory. When exiting the ruby-process and starting a new process docker-compose-api obviously doesn't know which containers it started before. This causes it to create/start a new container when requested.

I think that using named containers (https://github.com/mauricioklein/docker-compose-api/issues/13) might solve this problem. However if incremental names are used it might be a bit harder to tackle...

I'd be happy to help you out with this as I'm building a ruby app which needs to manage a bunch of containers.

[mauricioklein]

@jeroenj I don't know if the correct behavior is compose keep track of containers created by a different process. In fact, this is the actual behavior of docker-compose: if you ask to lift the same environment in two different terminals, you'll have all containers duplicated.

If you wanna keep track of those containers, you can always manipulate them directly from docker client.

The purpose of this gem is to be "in compliance" with original docker-compose.

Please let me know if you share the same opinion. Otherwise, I'm always open to suggestions.

Thanks in advance for improving this project!

[jeroenj]

What I actually meant is that - as far as I know - docker-compose (the utility) uses container names instead of id's to track what it has created and what the state is.

My current use case is a small web app that would use docker-compose-api to manage some containers. When restarting the app (e.g. after a deploy) docker-compose-api would lose track of what has been created/started. If container names would be used we could identify currently running containers and map them to the services defined in the compose.yml.

But you are correct that I could be using docker's api directly. That's actually what I'm doing right now. But the advantage of using compose is that it has support for dependencies (through linked containers), etc. which I now would have to work around manually.

Let me know if anything isn't clear. I've been investigating/prototyping quite a lot the last few days. I think I need some sleep. :)

[mauricioklein]

I understand your needs, however the default behavior of docker-compose utility is to not keep tracking of started containers. Well, this is what I know about docker compose, so, I might be wrong.

I'll research about the expected behavior of compose on your case. If it keeps track of started containers, we surely will add support to it in this gem.

I'll let you know what I discovery.

Thanks!

[jeroenj]

As far as I know it does. If I restart a docker host (and my containers are configured with restart: always) and I use docker-compose it knows the state of the defined services: the containers that are restarted are listed as running when using docker-compose. As far as I know it does this by tracking the container's name.

[mauricioklein]

Ok, I'll make some tests here to understand the real behavior of docker compose in this situation.

If the correct behavior is to keep track of active containers, that will be implemented.

Anyway, I have implemented the container name feature. I'll just made some further tests this evening and, until EOD I'll release a new version with this feature.

Then, I'll start checking the ability to keep track of containers started previously.

Please let me know if I can help in something else in the meantime.

Thanks!

[jeroenj]

Awesome, thanks! :+1: :smile:

[mauricioklein]

@jeroenj I've checked and, yes, after starting a whole environment with compose, running docker-compose ps in a new terminal it recognizes the containers started before.

I'm assign this issue as an enhancement and will start working on it.

As soon I have a solution implemented, I'll let you know.

Thanks!

[jeroenj]

:ok_hand: :+1:

[mauricioklein]

@jeroenj Please check Issue#13. Compose now supports container naming from compose file.

About this issue, it will be implemented, but probably will be available in a major release. So, I can't give you a deadline to such implementation, sorry.

Once the work is done and the major version is available, I'll let you know.

Thanks for contributing to project!

[jeroenj]

Thanks a lot! If I find something missing for my current setup I'd be happy to contribute. :)

[jeroenj]

Apparantly docker-compose uses labels instead of names to identify containers: https://github.com/docker/compose/pull/1356.

That aside the names attribution was something good. (using it right now).

We might look into using labels ourselves for the gem. Currently I have my setup working using names (will push the code later on). But I might look into a full implementation if we decide to go through with this prototype project I'm working on (which we'll probably do).

[jeroenj]

Ideally we use the exact same label-formatting as docker-compose does. By doing that we would be able to use both docker-compose (tool) and this gem along each other.

[mauricioklein]

Hello @jeroenj !

I've opened a PR with the feature of compose 'memory', it means, the ability to recognize and reload containers started previously on a new compose context.

I would like to invite you to review my PR, build yourself the gem and test it, to make sure I didn't forget anything on implementation/tests.

You can access the PR here.

If you have any concerns or suggestions, feel free to comment the PR.

Thanks in advance!

---

PS: Sorry about the long delay to provide such feature. I was really busy with my thesis :/

[jeroenj]

Awesome. I'll try to go through it this week.

An no worries on the timeframe. You're not obligated to implement features or fixes to me or anyone else. We should be happy for the work and effort you've already put into this. :+1:

[mauricioklein]

PR#22 merged, adding support to compose memory.

Gem version 1.1.0 released with these enhancements.

Thanks for all feedback and feel free to report problems and/or bugs related to this functionality.

[jeroenj]

Awesome. Thanks! :+1: