remind101 / conveyor

A fast build system for Docker images.
http://engineering.remind.com/introducing-conveyor/
BSD 2-Clause "Simplified" License
221 stars 24 forks source link

Persist builds and artifacts #52

Closed ejholmes closed 8 years ago

ejholmes commented 8 years ago

First commit is Godeps, so just look at the second commit.

This is the first step towards a build API. Historically, Conveyor has been almost entirely stateless, but if we want to provide an API that can be consumed to list Docker image artifacts for commits, we'll need to store those somewhere. This also now prevents multiple concurrent builds for the same git sha, which is almost never what we want (should cancel the previous build, then re-trigger it).

This is almost entirely backwards compatible, the only difference is that you now need to pass a --db flag (or DATABASE_URL env var) with a postgres connection string.

phobologic commented 8 years ago

This looks good - the only real concern I have is that using postgres here seems pretty heavy weight. I may have missed it, but for the most part this whole thing looks like postgres is being used as a k/v store, in which case something like Dynamo is a lot easier to maintain/setup.

ejholmes commented 8 years ago

I was thinking about it. Only reason I didn't is:

  1. I'm relying on transactions.
  2. I want ensure, at the db level, that we never have more than 1 build that's in process for any given sha. Doesn't look like you can define uniqueness constraints like that with dynamodb?
  3. Right now, Conveyor doesn't require that you run it on AWS.

And mostly, I haven't used dynamodb for anything else yet and I'm just more comfortable with Postgres for most things.