tar: invalid tar magic error

[ndd1970]

We are running the following script to create a docker image....but we get a "tar: invalid tar magic" as part of our output. Please see the full script and the output below. We have tried to use version of the gcc 2.29-r0 to 2.31-r0 but this made no difference, and we also went from Alpine 3.9 to Alpine 3.12.

FROM alpine:3.9

RUN apk upgrade -U \
    && echo "DEBUG *************** Past 1" \
    && apk --update add --no-cache ca-certificates curl openssl binutils wget xz \
    && echo "DEBUG *************** Past 2" \
    && GLIBC_VER="2.29-r0" \
    && echo "DEBUG *************** Past 3" \
    && GLIBC_CERTS_REPO="https://alpine-pkgs.sgerrand.com/" \
    && echo "DEBUG *************** Past 4" \
    && ALPINE_GLIBC_REPO="https://github.com/sgerrand/alpine-pkg-glibc/releases/download" \
    && echo "DEBUG *************** Past 5" \
    && wget -q -O /etc/apk/keys/sgerrand.rsa.pub ${GLIBC_CERTS_REPO}/sgerrand.rsa.pub \
    && echo "DEBUG *************** Past 6" \
    && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}.apk \
    && echo "DEBUG *************** Past 7" \
    && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}-bin.apk \
    && echo "DEBUG *************** Past 8" \
    && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-i18n-${GLIBC_VER}.apk > /tmp/i18n-${GLIBC_VER}.apk \
    && echo "DEBUG *************** Past 9" \
    && apk add /tmp/${GLIBC_VER}.apk \
    && echo "DEBUG *************** Past 10" \
    && apk add /tmp/${GLIBC_VER}-bin.apk \
    && echo "DEBUG *************** Past 11" \
    && apk add /tmp/i18n-${GLIBC_VER}.apk \
    && echo "DEBUG *************** Past 12" \
    && curl -Ls https://www.archlinux.org/packages/core/x86_64/gcc-libs/download > /tmp/gcc-libs.tar.xz \
    && echo "DEBUG *************** Past 13" \
    && mkdir /tmp/gcc \
    && echo "DEBUG *************** Past 14" \
    && ls /tmp -AlR \
    && echo "DEBUG *************** Past 14A - END OF DIRECTORY /tmp" \
    && tar -t /tmp/gcc-libs.tar.xz \
    && echo "DEBUG *************** Past 14B - END OF TAR DUMP " \
    && tar -xf /tmp/gcc-libs.tar.xz -C /tmp/gcc \
    && echo "DEBUG *************** Past 15" \
    && mv /tmp/gcc/usr/lib/libgcc* /tmp/gcc/usr/lib/libstdc++* /usr/glibc-compat/lib \
    && echo "DEBUG *************** Past 16" \
    && strip /usr/glibc-compat/lib/libgcc_s.so.* /usr/glibc-compat/lib/libstdc++.so* \
    && echo "DEBUG *************** Past 17" \
    && curl -Ls https://www.archlinux.org/packages/core/x86_64/zlib/download > /tmp/libz.tar.xz \
    && echo "DEBUG *************** Past 18" \
    && mkdir /tmp/libz \
    && echo "DEBUG *************** Past 19" \
    && tar -xf /tmp/libz.tar.xz -C /tmp/libz \
    && echo "DEBUG *************** Past 20" \
    && mv /tmp/libz/usr/lib/libz.so* /usr/glibc-compat/lib \
    && echo "DEBUG *************** Past 21" \
    && apk del binutils \
    && echo "DEBUG *************** Past 22" \
    && rm -rf /tmp/${GLIBC_VER}.apk /tmp/i18n-${GLIBC_VER}.apk /tmp/gcc /tmp/gcc-libs.tar.xz /tmp/libz /tmp/libz.tar.xz /var/cache/apk/* \
    && echo "DEBUG *************** Past 23" \
    && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
    && echo "DEBUG *************** Past 24" 

ENV LANG=en_US.UTF-8 \
    LANGUAGE=en_US.UTF-8

however, we get the following error in our output

+ docker build -t company-base:alpine-b708 .
Sending build context to Docker daemon  4.608kB
Step 1/3 : FROM alpine:3.9
3.9: Pulling from library/alpine
31603596830f: Pulling fs layer
31603596830f: Verifying Checksum
31603596830f: Download complete
31603596830f: Pull complete
Digest: sha256:414e0518bb9228d35e4cd5165567fb91d26c6a214e9c95899e1e056fcd349011
Status: Downloaded newer image for alpine:3.9
 ---> 78a2ce922f86

Step 2/3 : RUN apk upgrade -U     && echo "DEBUG *************** Past 1"     && apk --update add --no-cache ca-certificates curl openssl binutils wget xz     && echo "DEBUG *************** Past 2"     && GLIBC_VER="2.29-r0"     && echo "DEBUG *************** Past 3"     && GLIBC_CERTS_REPO="https://alpine-pkgs.sgerrand.com/"     && echo "DEBUG *************** Past 4"     && ALPINE_GLIBC_REPO="https://github.com/sgerrand/alpine-pkg-glibc/releases/download"     && echo "DEBUG *************** Past 5"     && wget -q -O /etc/apk/keys/sgerrand.rsa.pub ${GLIBC_CERTS_REPO}/sgerrand.rsa.pub     && echo "DEBUG *************** Past 6"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}.apk     && echo "DEBUG *************** Past 7"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}-bin.apk     && echo "DEBUG *************** Past 8"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-i18n-${GLIBC_VER}.apk > /tmp/i18n-${GLIBC_VER}.apk     && echo "DEBUG *************** Past 9"     && apk add /tmp/${GLIBC_VER}.apk     && echo "DEBUG *************** Past 10"     && apk add /tmp/${GLIBC_VER}-bin.apk     && echo "DEBUG *************** Past 11"     && apk add /tmp/i18n-${GLIBC_VER}.apk     && echo "DEBUG *************** Past 12"     && curl -Ls https://www.archlinux.org/packages/core/x86_64/gcc-libs/download > /tmp/gcc-libs.tar.xz     && echo "DEBUG *************** Past 13"     && mkdir /tmp/gcc     && echo "DEBUG *************** Past 14"     && ls /tmp -AlR     && echo "DEBUG *************** Past 14A - END OF DIRECTORY /tmp"     && tar -t /tmp/gcc-libs.tar.xz     && echo "DEBUG *************** Past 14B - END OF TAR DUMP "     && tar -xfv /tmp/gcc-libs.tar.xz -C /tmp/gcc     && echo "DEBUG *************** Past 15"     && mv /tmp/gcc/usr/lib/libgcc* /tmp/gcc/usr/lib/libstdc++* /usr/glibc-compat/lib     && echo "DEBUG *************** Past 16"     && strip /usr/glibc-compat/lib/libgcc_s.so.* /usr/glibc-compat/lib/libstdc++.so*     && echo "DEBUG *************** Past 17"     && curl -Ls https://www.archlinux.org/packages/core/x86_64/zlib/download > /tmp/libz.tar.xz     && echo "DEBUG *************** Past 18"     && mkdir /tmp/libz     && echo "DEBUG *************** Past 19"     && tar -xf /tmp/libz.tar.xz -C /tmp/libz     && echo "DEBUG *************** Past 20"     && mv /tmp/libz/usr/lib/libz.so* /usr/glibc-compat/lib     && echo "DEBUG *************** Past 21"     && apk del binutils     && echo "DEBUG *************** Past 22"     && rm -rf /tmp/${GLIBC_VER}.apk /tmp/i18n-${GLIBC_VER}.apk /tmp/gcc /tmp/gcc-libs.tar.xz /tmp/libz /tmp/libz.tar.xz /var/cache/apk/*     && echo "DEBUG *************** Past 23"     && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8     && echo "DEBUG *************** Past 24"

 ---> Running in 580eeaabdcbb
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/1) Upgrading ca-certificates-cacert (20191127-r0 -> 20191127-r2)
OK: 6 MiB in 14 packages

DEBUG *************** Past 1
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/10) Installing binutils (2.31.1-r2)
(2/10) Installing ca-certificates (20191127-r2)
(3/10) Installing nghttp2-libs (1.35.1-r2)
(4/10) Installing libssh2 (1.9.0-r1)
(5/10) Installing libcurl (7.64.0-r3)
(6/10) Installing curl (7.64.0-r3)
(7/10) Installing openssl (1.1.1g-r0)
(8/10) Installing wget (1.20.3-r0)
(9/10) Installing xz-libs (5.2.4-r0)
(10/10) Installing xz (5.2.4-r0)
Executing busybox-1.29.3-r10.trigger
Executing ca-certificates-20191127-r2.trigger
OK: 19 MiB in 24 packages

DEBUG *************** Past 2
DEBUG *************** Past 3
DEBUG *************** Past 4
DEBUG *************** Past 5
DEBUG *************** Past 6
DEBUG *************** Past 7
DEBUG *************** Past 8
DEBUG *************** Past 9

(1/1) Installing glibc (2.29-r0)
OK: 23 MiB in 25 packages
DEBUG *************** Past 10

(1/2) Installing libgcc (8.3.0-r0)
(2/2) Installing glibc-bin (2.29-r0)
Executing glibc-bin-2.29-r0.trigger
OK: 25 MiB in 27 packages
DEBUG *************** Past 11

(1/1) Installing glibc-i18n (2.29-r0)
OK: 51 MiB in 28 packages
DEBUG *************** Past 12
DEBUG *************** Past 13
DEBUG *************** Past 14

/tmp:
total 42716
-rw-r--r--    1 root     root        986988 Aug  5 12:31 2.29-r0-bin.apk
-rw-r--r--    1 root     root       2054960 Aug  5 12:31 2.29-r0.apk
drwxr-xr-x    2 root     root          4096 Aug  5 12:31 gcc
-rw-r--r--    1 root     root      33041312 Aug  5 12:31 gcc-libs.tar.xz
-rw-r--r--    1 root     root       7650415 Aug  5 12:31 i18n-2.29-r0.apk

/tmp/gcc:
total 0

DEBUG *************** Past 14A - END OF DIRECTORY /tmp

tar: short read

tar: /tmp/gcc-libs.tar.xz: not found in archive

The command '/bin/sh -c apk upgrade -U     && echo "DEBUG *************** Past 1"     && apk --update add --no-cache ca-certificates curl openssl binutils wget xz     && echo "DEBUG *************** Past 2"     && GLIBC_VER="2.29-r0"     && echo "DEBUG *************** Past 3"     && GLIBC_CERTS_REPO="https://alpine-pkgs.sgerrand.com/"     && echo "DEBUG *************** Past 4"     && ALPINE_GLIBC_REPO="https://github.com/sgerrand/alpine-pkg-glibc/releases/download"     && echo "DEBUG *************** Past 5"     && wget -q -O /etc/apk/keys/sgerrand.rsa.pub ${GLIBC_CERTS_REPO}/sgerrand.rsa.pub     && echo "DEBUG *************** Past 6"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}.apk     && echo "DEBUG *************** Past 7"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}-bin.apk     && echo "DEBUG *************** Past 8"     && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-i18n-${GLIBC_VER}.apk > /tmp/i18n-${GLIBC_VER}.apk     && echo "DEBUG *************** Past 9"     && apk add /tmp/${GLIBC_VER}.apk     && echo "DEBUG *************** Past 10"     && apk add /tmp/${GLIBC_VER}-bin.apk     && echo "DEBUG *************** Past 11"     && apk add /tmp/i18n-${GLIBC_VER}.apk     && echo "DEBUG *************** Past 12"     && curl -Ls https://www.archlinux.org/packages/core/x86_64/gcc-libs/download > /tmp/gcc-libs.tar.xz     && echo "DEBUG *************** Past 13"     && mkdir /tmp/gcc     && echo "DEBUG *************** Past 14"     && ls /tmp -AlR     && echo "DEBUG *************** Past 14A - END OF DIRECTORY /tmp"     && tar -t /tmp/gcc-libs.tar.xz     && echo "DEBUG *************** Past 14B - END OF TAR DUMP "     && tar -xfv /tmp/gcc-libs.tar.xz -C /tmp/gcc     && echo "DEBUG *************** Past 15"     && mv /tmp/gcc/usr/lib/libgcc* /tmp/gcc/usr/lib/libstdc++* /usr/glibc-compat/lib     && echo "DEBUG *************** Past 16"     && strip /usr/glibc-compat/lib/libgcc_s.so.* /usr/glibc-compat/lib/libstdc++.so*     && echo "DEBUG *************** Past 17"     && curl -Ls https://www.archlinux.org/packages/core/x86_64/zlib/download > /tmp/libz.tar.xz     && echo "DEBUG *************** Past 18"     && mkdir /tmp/libz     && echo "DEBUG *************** Past 19"     && tar -xf /tmp/libz.tar.xz -C /tmp/libz     && echo "DEBUG *************** Past 20"     && mv /tmp/libz/usr/lib/libz.so* /usr/glibc-compat/lib     && echo "DEBUG *************** Past 21"     && apk del binutils     && echo "DEBUG *************** Past 22"     && rm -rf /tmp/${GLIBC_VER}.apk /tmp/i18n-${GLIBC_VER}.apk /tmp/gcc /tmp/gcc-libs.tar.xz /tmp/libz /tmp/libz.tar.xz /var/cache/apk/*     && echo "DEBUG *************** Past 23"     && /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8     && echo "DEBUG *************** Past 24"' returned a non-zero code: 1

script returned exit code 1

[sgerrand]

I suspect the version of tar that you're using is unable to handle the compression format used by https://www.archlinux.org/packages/core/x86_64/gcc-libs/download.

[sgerrand]

To be clear, the error you're facing has nothing to do with the glibc Alpine Linux packages. It's caused by attempting to decompress a /tmp/gcc-libs.tar.xz file using tar with incorrect flags for that kind of compressed archive.

[sgerrand]

Using file to determine the file type should give you some clues as to how this can be resolved:

$ file /tmp/gcc-libs.tar.xz 
/tmp/gcc-libs.tar.xz: Zstandard compressed data (v0.8+), Dictionary ID: None

[sgerrand]

Making the following changes to your Dockerfile should fix this issue for you:

diff --git a/Dockerfile b/Dockerfile
index 00dd40d..0e5359a 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -2,7 +2,7 @@ FROM alpine:3.9

 RUN apk upgrade -U \
     && echo "DEBUG *************** Past 1" \
-    && apk --update add --no-cache ca-certificates curl openssl binutils wget xz \
+    && apk --update add --no-cache ca-certificates curl openssl binutils wget zstd \
     && echo "DEBUG *************** Past 2" \
     && GLIBC_VER="2.29-r0" \
     && echo "DEBUG *************** Past 3" \
@@ -24,15 +24,16 @@ RUN apk upgrade -U \
     && echo "DEBUG *************** Past 11" \
     && apk add /tmp/i18n-${GLIBC_VER}.apk \
     && echo "DEBUG *************** Past 12" \
-    && curl -Ls https://www.archlinux.org/packages/core/x86_64/gcc-libs/download > /tmp/gcc-libs.tar.xz \
+    && curl -Ls https://www.archlinux.org/packages/core/x86_64/gcc-libs/download > /tmp/gcc-libs.tar.zst \
     && echo "DEBUG *************** Past 13" \
     && mkdir /tmp/gcc \
     && echo "DEBUG *************** Past 14" \
     && ls /tmp -AlR \
     && echo "DEBUG *************** Past 14A - END OF DIRECTORY /tmp" \
-    && tar -t /tmp/gcc-libs.tar.xz \
+    && zstd -d /tmp/gcc-libs.tar.zst \
+    && tar -tf /tmp/gcc-libs.tar \
     && echo "DEBUG *************** Past 14B - END OF TAR DUMP " \
-    && tar -xf /tmp/gcc-libs.tar.xz -C /tmp/gcc \
+    && tar -xf /tmp/gcc-libs.tar -C /tmp/gcc \
     && echo "DEBUG *************** Past 15" \
     && mv /tmp/gcc/usr/lib/libgcc* /tmp/gcc/usr/lib/libstdc++* /usr/glibc-compat/lib \
     && echo "DEBUG *************** Past 16" \