Docker build error: No compiler found, expected minor version match with ghc-8.10.4 (x86_64)

[rhyslbw]

What Git revision are you using?

b3eb7596661be2a5d013a321f0dc8fbec781d492

What operating system are you using, and which version?

• [x] Linux / Ubuntu

Describe what the problem is?

make docker fails with:

~Specified file "README.md" for extra-source-files does not exist Specified file "CHANGELOG.md" for extra-source-files does not exist Specified source-dir "test/unit" does not exist~

No compiler found, expected minor version match with ghc-8.10.4 (x86_64) (based on resolver setting in /build/stack.yaml).
To install the correct GHC into /root/.stack/programs/x86_64-linux/, try running "stack setup" or use the "--install-ghc" flag. To use your system GHC installation, run "stack config set system-ghc --global true", or use the "--system-ghc" flag.
The command '/bin/sh -c stack install --flag "ogmios:production"' returned a non-zero code: 1
make: *** [Makefile:29: docker] Error 1

$ make docker
docker build server
Sending build context to Docker daemon  1.368MB
Step 1/15 : FROM ktorz/ogmios:snapshot as build
 ---> 933bf24a02b8
Step 2/15 : RUN stack build --only-dependencies
 ---> Using cache
 ---> dafc77be7dfb
Step 3/15 : COPY . .
 ---> Using cache
 ---> 2532cc126507
Step 4/15 : RUN stack install --flag "ogmios:production"
 ---> Running in faff86b1af64
<<<<<<<<<<<<<<<<<<
You are currently using Stack version 2.1.3, but version 2.5.1.1 is available
You can try to upgrade by running 'stack upgrade'
Tired of seeing this? Add 'recommend-stack-upgrade: false' to /root/.stack/config.yaml
>>>>>>>>>>>>>>>>>>
Cloning 75ed0d049c33274a6cb4c36c8538d4bf2ef9c30e from https://github.com/KtorZ/hjsonpointer
Cloning fde6e676f79f3f3320a558f20492ad816a2543a7 from https://github.com/KtorZ/hjsonschema
Cloning d74b39683792649c01113f40bf57724dcf95c96a from https://github.com/KtorZ/wai-routes
Specified file "README.md" for extra-source-files does not exist
Specified file "CHANGELOG.md" for extra-source-files does not exist
Specified source-dir "test/unit" does not exist
Stack has not been tested with GHC versions above 8.6, and using 8.10.4, this may fail
No compiler found, expected minor version match with ghc-8.10.4 (x86_64) (based on resolver setting in /build/stack.yaml).
To install the correct GHC into /root/.stack/programs/x86_64-linux/, try running "stack setup" or use the "--install-ghc" flag. To use your system GHC installation, run "stack config set system-ghc --global true", or use the "--system-ghc" flag.
The command '/bin/sh -c stack install --flag "ogmios:production"' returned a non-zero code: 1
make: *** [Makefile:29: docker] Error 1

What should be the expected behavior?

Build completes

[KtorZ]

I don't think the missing extra-source-files have anything to do with this. These are mere warnings because stack can't find those declared files, though they were omitted purposely from the docker image (they aren't needed to build the source).

What the error suggests is that the snapshot you're using doesn't know about GHC-8.10.4; I've updated the snapshot some time ago so a docker pull ktorz/ogmios:snapshot should fix the problem. Perhaps, I should put the snapshot on a different repository and use unique tags for them to force automatic re-pull when snapshot changes :thinking: ... This is not the first time this happens and also got me from time to time.

[rhyslbw]

Ah you're right, I misread the error log 🤦

I'm curious about the need to have a separate image. Can we not just rely on Docker caching to avoid this complexity?

[KtorZ]

Yes, the original intent I guess was to give people a fast first docker build and have the parent image contain pre-built dependencies (takes about 45minutes to build...). So that locally, you'd only have to rebuild the sources. I guess it's a quite minor victory, because indeed, it's only a one-time cost and then, Docker caching kicks in and you get the same benefits, but at least, when the dependencies changes in the Dockerfile like now, you get to rebuild them automatically.

[rhyslbw]

We can actually still use Docker Hub as a remote cache provider via a native Docker feature. I implemented this in cardano-rosetta:

https://github.com/input-output-hk/cardano-rosetta#with-local-source-code

[KtorZ]

:exploding_head: TIL.

[KtorZ]

I've revised the docker setup to only require a single Dockerfile, basically merging the 'snapshot' and the main one. Also took the opportunity to do some fixes in the Dockerfile itself since it was... not building since I upgraded to GHC-8.10.4.

The full build take about 30 minutes on my machine, and it can be shrunk down by re-using the caching from dockerhub. The build stage is also run in two phases, so that adjusting the source code doesn't require a full build (however, any change in the dependencies will).

I've added a notice in the documentation about using --cache-from: https://ogmios.dev/getting-started/docker/ and also made it by default in the docker-compose declaration:

https://github.com/KtorZ/cardano-ogmios/blob/master/docker-compose.yml#L21-L23

I'll wait for the build on Dockerhub to finish to be sure everything is now okay before closing this ticket.