Open Riscky opened 1 year ago
@Riscky aside from the points mentioned above, could you give a list of things that need to be done in order for this PR to be ready for review/deploy? I would like to know what the next steps are that need to be taken, and how they are best taken.
@Riscky aside from the points mentioned above, could you give a list of things that need to be done in order for this PR to be ready for review/deploy? I would like to know what the next steps are that need to be taken, and how they are best taken.
I think the most important question to solve is how to bring rails
and other service commands in scope. Putting everything in a Docker containers is nice for running the application, but when trying to run service commands (eg. rails db:setup
), it requires you to run something like docker-compose exec koala-development -- rails db:migrate
against a running container. A big drawback of using docker-compose exec
(or the docker
equivalent) is there is more things that can break, and feedback is rather minimal.
The 'simple' solution is to install everything on the host directly. This was the approach we took before we started using Nix, and made setup on a developer machine quite tedious.
If we want to truly replace Nix, we need a development shell, which, like Nix, brings the necessary dependencies in scope.
We also need a way of building the application so we can put it on the server batteries included.
Docker only covers the latter (as does Nix).
We also need an easy way of starting all services in a development environment, which in this PR is covered by docker-compose
, and isn't covered by Nix.
We can use Procfile's and some runner like hivemind
for that too, so we don't need to use containers to get to that goal.
Simply put, in the current state this PR solves some of the problems we have (Nix is hard to learn, we don't have a single entry point to run the stack), but introduces new problems (no easy way to run service commands).
What is the best way forward?
I think we should keep Nix as a developer shell, as it makes it very easy to get the right tooling to all developers.
We should however rip out the stuff that doesn't work, like pinning all NodeJS dependencies and gems with Nix.
That means devs have to run bundle install
still to get their shells working, after starting a Nix shell.
We should keep Docker for running postgres en redis (like we already did), and for running the webpack dev server and Koala itself.
So: service commands in nix shell
, running the stack with docker-compose
, shipping to the server as a Docker container.
This keeps the Nix setup to a very manageable level, makes it easy to run service commands and the stack, and makes the build self-contained.
To get there, based on this PR, we'll need to:
default.nix
with just the toplevel tooling. Basically just the things that are now in propagatedBuildInputs
+ Docker + docker-composewebpack-dev-server
sidekiq
and connect Koala to itHad some time to look at this again, short summary of the current status:
webpack-dev-server
is not dockerized, because it seems to hate us. We don't need it;Apparently, docker's network_mode host doesn't work on windows and mac, so we probably definitely need to switch to normal networks again (https://docs.docker.com/network/drivers/host/)
This PR throws our wonky, broken Nix setup out of the window, and replaces it with Docker. Now I hear you thinking: "these are not even the same category of software products", and you would be right. Docker doesn't replace Nix in terms of dependency management, instead we go back to good old apt+bundler+yarn. But instead of making developers install these pieces of software themselves, we create a nice Docker image that contains all of these. This makes sure that every developer runs (mostly) the same versions of the dependencies. It makes setting up Koala a lot easier.
Note that this PR is very much Work In Progress. The production container is building just fine, but:
webpack-dev-server
container is not doing its job (if you enable it, Koala tries to connect to it, but fails)sidekiq
is still missingThe CI pipeline is brokenI fixed the CI pipeline, and added a second one that can build and publish the docker container. The latter needs testing@Mstiekema, @TobiasDeBruijn, you both expressed interest in helping me out with this PR, so here goes :)