search

redhat-actions / buildah-build

GitHub Action to use 'buildah' to build a container image.
https://github.com/marketplace/actions/buildah-build
MIT License
137 stars 35 forks source link

[QUESTION] failing with buildah 1.20.0 (ubuntu-latest) #45

Closed jpopelka closed 3 years ago

jpopelka commented 3 years ago

Question

Hello, thank you for this action, it's marvelous. Today our builds started to fail and I'm wondering what's changed and where can be the cause.

Writing manifest to image destination
Storing signatures
level=error msg="error unmounting /home/runner/.local/share/containers/storage/overlay/0781b6291aebaf9c44638d47503ccc8967e46e541203bd35d79bef4a0566b5ac/merged: invalid argument"
error mounting new container: error mounting build container "b11e0c122471549c9c260843257e96e91b92ed343041b40f9f119667b6979a0f": error creating overlay mount to /home/runner/.local/share/containers/storage/overlay/0781b6291aebaf9c44638d47503ccc8967e46e541203bd35d79bef4a0566b5ac/merged, mount_data=",lowerdir=/home/runner/.local/share/containers/storage/overlay/l/JVFKZCGLHI42F7Y324TMXFQZ4P:/home/runner/.local/share/containers/storage/overlay/l/3PUMEEVI6PIVBZDO2HZF7SXLWO,upperdir=/home/runner/.local/share/containers/storage/overlay/0781b6291aebaf9c44638d47503ccc8967e46e541203bd35d79bef4a0566b5ac/diff,workdir=/home/runner/.local/share/containers/storage/overlay/0781b6291aebaf9c44638d47503ccc8967e46e541203bd35d79bef4a0566b5ac/work,userxattr": invalid argument
level=error msg="exit status 125"

The only difference between the failing one and 20hours old passing previous one I see in logs is the buildah version 1.19.8 (previous, OK) vs. 1.20.0 (failed).

I know it's most likely not your fault, that's why I'm creating a question in case someone else also hits this.

jpopelka commented 3 years ago

Hmm, this still says that ubuntu-latest contains "Buildah 1.19.8".

divyansh42 commented 3 years ago

Yeah, they should update that.

tetchel commented 3 years ago

same thing here? https://github.com/redhat-actions/buildah-build/pull/47

Good thing we have the version output

divyansh42 commented 3 years ago

same thing here? #47

Good thing we have the version output

Yes

divyansh42 commented 3 years ago

FYI: https://github.com/containers/buildah/issues/3120

divyansh42 commented 3 years ago

Possible workaround as per buildah troubleshooting that we have did to solve this problem temporarily. 

- run: |
    sudo apt-get install fuse-overlayfs
    mkdir -vp ~/.config/containers
    printf "[storage.options]\nmount_program=\"/usr/bin/fuse-overlayfs\"" > ~/.config/containers/storage.conf
tetchel commented 3 years ago

For anyone else coming into this issue, you can also work around this by using runs-on: ubuntu-18.04 for the job that runs buildah. So you'll get an older, working version of buildah while we look for a permanent solution.

Or you can use the workaround above

edit: does not sound like GitHub will fix this on their end, either. https://github.com/actions/virtual-environments/issues/3080

edit2: fixed in buildah-build@v2

tetchel commented 3 years ago

this is fixed in v2 / v2.4 now.

tetchel commented 3 years ago

if anyone is coming here because they see the warning printed on an ubuntu-18.04 system, don't worry, the action will still work.

however it is recommended to install fuse-overlayfs for better fs performance. upgrading to 20.04 will also remove the warning