search

awesome-kyuubi / hadoop-testing

Testing Sandbox for Hadoop Ecosystem Components
Apache License 2.0
32 stars 13 forks source link

Remove and clean old images and containers before invoke re-build image? #17

Closed yanghua closed 10 months ago

pan3793 commented 10 months ago

in fact, it's by design to leverage the docker build cache ...

but i'm not sure how does docker cache mechanism works with ADD file command, can it identify the file content changes while the filename is the same?

yanghua commented 10 months ago

but i'm not sure how does docker cache mechanism works with ADD file command, can it identify the file content changes while the filename is the same?

Not sure too.

Share my scenario, I am building hadoop-master docker image for Hudi, and it caused some errors(and the whole environment is chaotic), so I must update the dockerfile again. For rollbacking to a pure environment, I only removed the hadoop-master image. When I rebuilt the hadoop-master and started the Hadoop cluster, all data nodes failed due to inconsistent env. So maybe a switch config option named RESET is helpful for this scenario?

However, doing the reset action is indeed dangerous in some scenarios where users have their useful data and personal config option and so on.

pan3793 commented 10 months ago

try ./build-image.sh --no-cache

pan3793 commented 10 months ago

try ./build-image.sh --no-cache

tested locally, works as expected