Ok let me lay out my high level thoughts here. What I was working towards:
Whole stack starts fast and strives to eliminate unnecessary work.
I found a few huge ugly points:
invoker has to run bundle update and setup_exercism_config
coordinator has to run bundle update and setup_exercism_config
website has to run bundle update and setup_exercism_config
So we pay this price 3 times. This does NOT need to be done in 3 places.
website is slow to run setup_aws_locally.rb which is needed for other processes (so they spew out errors for a while)
This change creates a new startup docker that moves all this "start the stack" complexity there... and because it's only running Ruby (not the whole Rails stack) it's quite fast... and if we use git tags (vs using Rubygems) for versioning then we avoid the SLOW bundle update in favor of simply pulling a new copy of code from git.
So to upgrade from exercism config 0.25 to 0.26 you can just bring up a new stack and that will happen as the stack comes online. If the dependencies ever change this will necessitate rebuilding the image, but that's as it should be. The images should be "ready to go" as much as possible.
I have a whole host of PRs along these lines but wanted to start the big picture discussion here... I've also pushed my changes to tooling invoker so you can see how it simplifies the components.
My whole stack starts much faster with a lot fewer errors with these changes.
You could also make an argument to move everything into v3_website (and I'm not opposed) but I think for now it's a bit easier to iterate on this way and it increases separation of concerns between pieces of the architecture because now I can bring up the whole tooling chain without firing up the website at all (since this moves the AWS setup out of the website's responsibility).
Ok let me lay out my high level thoughts here. What I was working towards:
Whole stack starts fast and strives to eliminate unnecessary work.
I found a few huge ugly points:
setup_aws_locally.rb
which is needed for other processes (so they spew out errors for a while)This change creates a new
startup
docker that moves all this "start the stack" complexity there... and because it's only running Ruby (not the whole Rails stack) it's quite fast... and if we use git tags (vs using Rubygems) for versioning then we avoid the SLOW bundle update in favor of simply pulling a new copy of code from git.So to upgrade from exercism config 0.25 to 0.26 you can just bring up a new stack and that will happen as the stack comes online. If the dependencies ever change this will necessitate rebuilding the image, but that's as it should be. The images should be "ready to go" as much as possible.
I have a whole host of PRs along these lines but wanted to start the big picture discussion here... I've also pushed my changes to tooling invoker so you can see how it simplifies the components.
My whole stack starts much faster with a lot fewer errors with these changes.
You could also make an argument to move everything into v3_website (and I'm not opposed) but I think for now it's a bit easier to iterate on this way and it increases separation of concerns between pieces of the architecture because now I can bring up the whole tooling chain without firing up the website at all (since this moves the AWS setup out of the website's responsibility).