Break up the monolith

[mattwynne]

The docker container here is huge, and is only getting bigger and bigger as each piece of Cucumber needs new dependencies.

Instead of having one docker image that can build anything, @ciaranmcnulty has suggested using smaller specialized container images for different environments - one for Java, one for JavaScript etc. These images would be easier to maintain as they'd be independent of each other.

There's a lot more to this proposal, but Ciaran has the details and the knowledge of how to implement in Docker. I just wanted to flag this up for feedback.

[gaeljw]

I thought the huge image was required for monorepo build/release at some point but I don't know much the monorepo.

If we can have smaller images, it's definitely a good move. Huge image is just asking for trouble in the long term: assuming we use the image in a free CI context someday (Github Actions for instance), downloading the gigas of the image would be a pain (it is already for me on my laptop!).

We can probably consider re-using trusted images with just small configuration changes if we go this way, so this would improve maintainability as well.

[mpkorstanje]

I have to say I'm a bit puzzled about this one.

• The docker image has to be able to build all languages in the monorepo in one go locally. So all languages must be supported in at least some image.

• The monorepo already uses smaller images for the parallel, single language, parts of the build.

  • I've intentionally chosen ubuntu as the base image. While not small or efficient, it's the only image where everything can be installed with minimal effort. It's been optimized to minimize the most scarce resource around. Our time.

Now I'm not opposed to anyone trying and haven't got the time to argue it through anyway but beware of these constraints. I would advise against any solution that optimizes image size by making the docker file more complex.

[ciaranmcnulty]

The docker image has to be able to build all languages in the monorepo in one go locally

Where does the constraint that this has to happen in one image come from?

[Ubuntu is] the only image where everything can be installed with minimal effort

Agreed, I think this is driven by the constraint above though and can be reassessed

The monorepo already uses smaller images for the parallel, single language, parts of the build

Oh, can you point out where please? This seems to conflict with your first statement BTW ;)

We'll make a proof of concept to outline the general approach with more, smaller, containers

[mpkorstanje]

Where does the constraint that this has to happen in one image come from?

The release process make a single release for all implementations at once.

Oh, can you point out where please? This seems to conflict with your first statement BTW ;)

There is a difference between building for CI and building for release.

https://github.com/cucumber/cucumber/blob/master/.circleci/config.yml

We'll make a proof of concept to outline the general approach with more, smaller, containers

I think you (and Matt too) should make a few releases in the monorepo first so you understand the process that you're looking to change. Granted this may be a bit difficult if all you have is M1 silicon.

[mattwynne]

We had a discussion about this today at the cukenspace. Here are the notes: https://miro.com/app/board/o9J_lHcvdzg=/?moveToWidget=3074457358129254287&cot=14

@ciaranmcnulty if we did have, e.g. one container for javascript, one container for jvm, etc, is there a way we could also still build out a monoimage, based on those smaller images?

[ciaranmcnulty]

is there a way we could also still build out a monoimage, based on those smaller images?

Not practically. Even if they have a common base image (e.g. alpine/ubuntu) you have to know which files to copy from each one