search

docker / for-linux

Docker Engine for Linux
https://docs.docker.com/engine/installation/
754 stars 85 forks source link

gettimeofday() is broken on an arm32v7 program running in a an arm64v8 image on an arm64v8 host #1145

Open TheBrokenRail opened 3 years ago

TheBrokenRail commented 3 years ago

Expected behavior

As time passes the values being reported from gettimeofday() increase.

Actual behavior

They stay at 0 forever.

Steps to reproduce the behavior

Use A Raspberry Pi 4 Running Raspbian Buster ARM64 (https://downloads.raspberrypi.org/raspios_arm64/images/)

$ docker run -it arm64v8/debian:bullseye
# dpkg --add-architecture armhf
# apt update && apt dist-upgrade -y && apt install -y build-essential nano gcc-arm-linux-gnueabihf libc6:armhf
# nano time.c
// #include <stdio.h>
// #include <stddef.h>
// #include <sys/time.h>
//
// int main() {
//     int i = 0;
//     struct timeval tv;
//     while (1) {
//         gettimeofday(&tv, NULL);
//         fprintf(stderr, "Loop Iteration: %i tv.tv_sec: %ld\n", i++, tv.tv_sec);
//     }
// }
# arm-linux-gnueabihf-gcc -o time time.c
# ./time
// Notice how the loop iteration keeps increasing, but tv.tv_sec stays at 0
# gcc -o time64 ./time.c
// Notice how bot the loop iteration and tv.tv_sec keep increasing

Output of docker version:

Client:
 Version:           18.09.1
 API version:       1.39
 Go version:        go1.11.6
 Git commit:        4c52b90
 Built:             Fri, 13 Sep 2019 10:45:43 +0100
 OS/Arch:           linux/arm64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.1
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.11.6
  Git commit:       4c52b90
  Built:            Fri Sep 13 09:45:43 2019
  OS/Arch:          linux/arm64
  Experimental:     false

Output of docker info:

Containers: 6
 Running: 0
 Paused: 0
 Stopped: 6
Images: 153
Server Version: 18.09.1
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 1.0.0~rc6+dfsg1-3
init version: v0.18.0 (expected: fec3683b971d9c3ef73f284f176672c44b448662)
Security Options:
 seccomp
  Profile: default
Kernel Version: 5.4.72-v8+
Operating System: Debian GNU/Linux 10 (buster)
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 912.4MiB
Name: raspberrypi
ID: J2DU:HAS3:4TUI:QB5M:37G5:T526:MMF7:XGMP:I24W:JBMF:V5NO:VTOF
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support

Additional environment details (AWS, VirtualBox, physical, etc.) Raspberry Pi 4

nobody5050 commented 3 years ago

Experiencing the same issue

TheBrokenRail commented 3 years ago

Also, this doesn't occur on an ARM32 program running on an ARM64 image on an x86_64 host using QEMU user-mode.

TheBrokenRail commented 3 years ago

The issue occurs with docker-ce as well as docker.io.

TheBrokenRail commented 3 years ago

The issue also occurs on Linux Kernel 5.10.

TheBrokenRail commented 3 years ago

It also occurs with an ARM32v7 program running in a ARM32v7 container on an ARM64v8 host.

justincormack commented 3 years ago

Does the same happen outside the container?

TheBrokenRail commented 3 years ago

No

thaJeztah commented 3 years ago

Are you still seeing this on docker 19.03? Thinking of https://github.com/moby/moby/pull/40769, which went into docker 19.03.9 and up

TheBrokenRail commented 3 years ago

I haven't tested yet, but I don't know why it would since it just uses the norm gettimeofday syscall, rather than one of the newer 64bit.variations. This issue occurs on newly compiled test code as well as code from 2013. Also, it works on x86_64 systems using QEMU user-mode.

TheBrokenRail commented 3 years ago

I am using Docker 19.03.14now. The issue is one when running with seccomp set to unconfirmed. Running strace with date produces:

execve("/bin/date", ["date"], 0xffb6cbf4 /* 12 vars */) = 0
brk(NULL)                               = 0x296d000
uname({sysname="Linux", nodename="raspberrypi", ...}) = 0
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=13212, ...}) = 0
mmap2(NULL, 13212, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf7b27000
close(3)                                = 0
openat(AT_FDCWD, "/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\3\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0Y\253\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=973416, ...}) = 0
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7b25000
mmap2(NULL, 1042632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf7a04000
mprotect(0xf7aee000, 61440, PROT_NONE)  = 0
mmap2(0xf7afd000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xe9000) = 0xf7afd000
mmap2(0xf7b01000, 6344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf7b01000
close(3)                                = 0
set_tls(0xf7b25c10)                     = 0
mprotect(0xf7afd000, 8192, PROT_READ)   = 0
mprotect(0xbea000, 4096, PROT_READ)     = 0
mprotect(0xf7b2c000, 4096, PROT_READ)   = 0
munmap(0xf7b27000, 13212)               = 0
brk(NULL)                               = 0x296d000
brk(0x298e000)                          = 0x298e000
clock_gettime64(CLOCK_REALTIME, 0xffda5cb0) = -1 EPERM (Operation not permitted)
openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118
_llseek(3, -62, [56], SEEK_CUR)         = 0
read(3, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 62
close(3)                                = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0
write(1, "Thu Jan  1 00:00:00 UTC 1970\n", 29Thu Jan  1 00:00:00 UTC 1970
) = 29
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++
TheBrokenRail commented 3 years ago

This is especially odd because that PR was supposed to whitelist clock_gettime64. It was merged into 19.03.9 and I am using Docker 19.03.14 from the official repo:

Docker version 19.03.14, build 5eb3275
TheBrokenRail commented 3 years ago 
$ docker version
Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275
 Built:             Tue Dec  1 19:20:45 2020
 OS/Arch:           linux/arm64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275
  Built:            Tue Dec  1 19:19:15 2020
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.3.9
  GitCommit:        ea765aba0d05254012b0b9e595e995c09186427f
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec368
TheBrokenRail commented 3 years ago

Reading through this (https://github.com/moby/moby/issues/40734) it seems the issue caused by Debian Buster's version of libseccomp doesn't recognize the newer syscalls and blocks them.