alpine:3.19.0 - Illegal instruction or CrashLoopBackOff

[g1franc]

Hello, we are having issue since version 3.19 and it was working fine in version 3.18.

When trying to instantiate image in K8S cluster, the pod keeps crashing with exit code 132.

When running the pod in interactive mode, every command return an "Illegal instruction" message (see details after).

Is alpine3.19 having new CPU requirements ?

kubectl -n my-namespace --image=alpine:3.19.0 --rm -it --restart=Never
If you don't see a command prompt, try pressing enter.
/ #
/ # id
Illegal instruction
/ # test
/ # who
Illegal instruction
/ # dmesg
Illegal instruction
/ # cat
Illegal instruction
/ # echo

/ # test
/ # echo $?
1
/ # more
Illegal instruction

Here is the output of command cat /proc/cpuinfo

kubectl -n my-namespace --image=alpine:3.18.0 --rm -it --restart=Never
If you don't see a command prompt, try pressing enter.
/ # cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 85
model name      : Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
stepping        : 7
microcode       : 0x5003302
cpu MHz         : 2095.077
cache size      : 28160 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
bugs            : spectre_v1 spectre_v2 spec_store_bypass swapgs itlb_multihit mmio_stale_data retbleed eibrs_pbrsb
bogomips        : 4190.15
clflush size    : 64
cache_alignment : 64
address sizes   : 45 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
.............

[nicoske]

Hi, I experience exactly the same issue.

Test image:

FROM alpine:3.19
RUN apk add --no-cache bash file
RUN echo -e '#!/bin/bash\n\
echo "System Information:";\n\
uname -a;\n\
echo "Architecture:";\n\
arch;\n\
echo "CPU Information:";\n\
cat /proc/cpuinfo;\n\
echo "Checking sleep command:";\n\
file $(which sleep);\n\
echo "Attempting to execute sleep:";\n\
sleep 5;\n\
echo "Hello World"' > /hello.sh
RUN chmod +x /hello.sh
ENTRYPOINT ["/hello.sh"]

Logs:

System Information:
/hello.sh: line 3:     7 Illegal instruction     uname -a
Architecture:
/hello.sh: line 5:     8 Illegal instruction     arch
CPU Information:
/hello.sh: line 7:     9 Illegal instruction     cat /proc/cpuinfo
Checking sleep command:
/hello.sh: line 9:    11 Illegal instruction     file $(which sleep)
Attempting to execute sleep:
/hello.sh: line 11:    12 Illegal instruction     sleep 5
Hello World

Works fine with 3.18:

System Information:
Linux alpinehello18-694b7c97bc-hk2sv 5.19.0-46-generic #47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 21 15:35:31 UTC 2 x86_64 Linux
Architecture:
x86_64
CPU Information:
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 85
model name  : Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
stepping    : 7
microcode   : 0x5003302
cpu MHz     : 2095.078
cache size  : 28160 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 22
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat pku ospke avx512_vnni md_clear flush_l1d arch_capabilities
bugs        : spectre_v1 spectre_v2 spec_store_bypass swapgs itlb_multihit mmio_stale_data retbleed eibrs_pbrsb
bogomips    : 4190.15
clflush size    : 64
cache_alignment : 64
address sizes   : 45 bits physical, 48 bits virtual
power management:

[snip]

Checking sleep command:
/bin/sleep: symbolic link to /bin/busybox
Attempting to execute sleep:
Hello World

[swat5421]

Check if you have dynatrace oneagent installed.

[ThibaultNocchi]

We have this error and we indeed have dynatrace agent. That seems so random, how is this related?

[swat5421]

oneagent injecting shared library into a processes and apparently it's incompatible with 3.19. After uninstall of oneagent the issue is disappears for me.

[ThibaultNocchi]

That makes sense, thanks for the info!

[g1franc]

I confirm the same behaviour:

• with Dynatrace -> illegal instruction
• without Dynatrace -> working as expected

[rbren]

According to these release notes, this might be solved with the latest version of dynatrace: https://docs.dynatrace.com/docs/whats-new/release-notes/oneagent/sprint-285

Regardless, can probably close this issue as it's clearly a dynatrace problem

[g1franc]

According to these release notes, this might be solved with the latest version of dynatrace: https://docs.dynatrace.com/docs/whats-new/release-notes/oneagent/sprint-285

Regardless, can probably close this issue as it's clearly a dynatrace problem

Seems legit to me. Thanks