ticktock cache issue

[alexcb]

a caching issue regarding the experimental ticktock buildkit fork was mentioned by @c0va23 in https://github.com/earthly/earthly/issues/2454#issuecomment-2137347470_

I just tried an Earthfile such as

VERSION 0.8

common:
  FROM alpine
  RUN echo in common && sleep 5 && echo done common

a:
  FROM +common
  RUN echo in a

b:
  FROM +common
  RUN echo in b

test:
  BUILD +a
  BUILD +b

but couldn't reproduce the issue. Any chance you have a minimal reproduction for this @c0va23?

[c0va23]

Today I tried to extract minimal snippets to reproduce this issue. But I cannot reproduce the issue on my development machine.  I think this issue is caused by the configuration of our CI.

For context, our Earthly configuration with cache issue looks like:

# common/Earthfile
VERSION 0.8

common:
  FROM jdk
  COPY dependencies ./
  RUN ./download.sh

# a/Earthfile
VERSION 0.8
check:
  FROM ../common+common
  RUN echo in a

# b/Earthfile
VERSION 0.8
check:
  FROM ../common+common
  RUN echo in b

# c/Earthfile
VERSION 0.8
check:
  FROM ../common+common
  RUN echo in c

On CI it was ran some time like:

git clone ...
earthly ./a+check
earthly ./b+check
earthly ./c+check

The cache miss could happen on COPY dependencies ./ line on any of earthly ./a+check call.

Today I replaced many earthly calls with ones that call other targets via BUILD. And this fixes the issue for our CI.

[alexcb]

I wonder if the common step was getting garbage-collected between the individual earthly calls?

[c0va23]

It could be GC. The builds with this issue wrote "last cleared" log with a non-zero value. But the logs from buildkitd daemon were lost. And I cannot confirm this.