[oci runtime error ] 500 Server Error upon Creation of Source Server

bhooten commented 7 years ago

Hello! I'm having issues while creating a new source server. Upon initial start of the server, this error appears.

    Error: (HTTP code 500) server error - invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n"
        at /srv/daemon/node_modules/docker-modem/lib/modem.js:229:17
        at getCause (/srv/daemon/node_modules/docker-modem/lib/modem.js:259:7)
        at Modem.buildPayload (/srv/daemon/node_modules/docker-modem/lib/modem.js:228:5)
        at IncomingMessage.<anonymous> (/srv/daemon/node_modules/docker-modem/lib/modem.js:204:14)
        at emitNone (events.js:91:20)
        at IncomingMessage.emit (events.js:185:7)
        at endReadableNT (_stream_readable.js:974:12)
        at _combinedTickCallback (internal/process/next_tick.js:74:11)
        at process._tickDomainCallback (internal/process/next_tick.js:122:9)

npm version returns the following.

  npm: '3.10.10',
  ares: '1.10.1-DEV',
  http_parser: '2.7.0',
  icu: '56.1',
  modules: '48',
  node: '6.9.4',
  openssl: '1.0.2j',
  uv: '1.9.1',
  v8: '5.1.281.89',
  zlib: '1.2.8' }

Thanks.

schrej commented 7 years ago

Could you please reformat that? Use three ` instead of one for code blocks and try to do something about that insane amount of slashes.

bhooten commented 7 years ago

Sorry @schrej, don't use GitHub issues a lot. I fixed it to a degree.

schrej commented 7 years ago

@Brad126735 No worries, it's much better now ;)

parkervcp commented 7 years ago

Any more info like OS/distro, docker version

bhooten commented 7 years ago

My apologies @parkervcp. Distro - CentOS 7.3.1611 Docker Version - 1.12.5, build 7392c3b

DaneEveritt commented 7 years ago

@Brad126735 Is there anymore of that error available? Like anything before or after?

bhooten commented 7 years ago

This is the full trace from daemon/logs. @DaneEveritt

19:59:40.938Z ERROR wings: (HTTP code 500) server error - invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n"  (server=9f08193d-c9f9-4c1c-a068-d40596a89156)
    Error: (HTTP code 500) server error - invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" 
     at /srv/daemon/node_modules/docker-modem/lib/modem.js:229:17
     at getCause (/srv/daemon/node_modules/docker-modem/lib/modem.js:259:7)
     at Modem.buildPayload (/srv/daemon/node_modules/docker-modem/lib/modem.js:228:5)
     at IncomingMessage.<anonymous> (/srv/daemon/node_modules/docker-modem/lib/modem.js:204:14)
     at emitNone (events.js:91:20)
     at IncomingMessage.emit (events.js:185:7)
     at endReadableNT (_stream_readable.js:974:12)
     at _combinedTickCallback (internal/process/next_tick.js:74:11)
     at process._tickDomainCallback (internal/process/next_tick.js:122:9)

DaneEveritt commented 7 years ago

Are you running the daemon as root?

There seems to be discussion of the issue in https://github.com/docker/docker/issues/28109

bhooten commented 7 years ago

Yes, the daemon is being run as root, should it be otherwise? @DaneEveritt

DaneEveritt commented 7 years ago

No, that is correct. Can you provide the docker logs themselves from when this occurs?

I believe CentOS is /var/log/daemon.log | grep docker but potentially journalctl -u docker.service.

bhooten commented 7 years ago

This is my current docker log.


Jan 16 06:37:21 ns352682.ip-91-121-85.eu systemd[1]: Starting Docker Application Container Engine...
Jan 16 06:37:22 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:22.328408524+01:00" level=info msg="libcontainerd: new containerd process, pid: 1249"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.384561720+01:00" level=warning msg="devmapper: Usage of loopback devices is strongly discouraged for production use. Please use `--storage-opt dm.thinpooldev` or use `man docker` to refer to dm.thinpooldev section."
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.403659176+01:00" level=warning msg="devmapper: Base device already exists and has filesystem xfs on it. User specified filesystem  will be ignored."
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.440454657+01:00" level=info msg="[graphdriver] using prior storage driver \"devicemapper\""
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.478096521+01:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.479486655+01:00" level=info msg="Loading containers: start."
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: ..........time="2017-01-16T06:37:23.504890312+01:00" level=info msg="Firewalld running: false"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.608408951+01:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.634451379+01:00" level=info msg="Loading containers: done."
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.634722542+01:00" level=info msg="Daemon has completed initialization"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.634757688+01:00" level=info msg="Docker daemon" commit=7392c3b graphdriver=devicemapper version=1.12.5
Jan 16 06:37:23 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:37:23.647470686+01:00" level=info msg="API listen on /var/run/docker.sock"
Jan 16 06:37:23 ns352682.ip-91-121-85.eu systemd[1]: Started Docker Application Container Engine.
Jan 16 06:38:25 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:25+01:00" level=info msg="Firewalld running: false"
Jan 16 06:38:33 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:33+01:00" level=info msg="Firewalld running: false"
Jan 16 06:38:40 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:40+01:00" level=info msg="Firewalld running: false"
Jan 16 06:38:44 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:44+01:00" level=info msg="Firewalld running: false"
Jan 16 06:38:56 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:56.271401777+01:00" level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" id=4ff26c8abd151766df6ea277e6433a35d87245023105057f9082f15781ba823b
Jan 16 06:38:56 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:56.272451762+01:00" level=error msg="Create container failed with error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 06:38:56 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:38:56.369492848+01:00" level=error msg="Handler for POST /containers/4ff26c8abd15/start returned error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/f1d9822a59be35362f9a5d7235f21b25083ede4ed718a0d56aed00f41ca66631/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 06:55:27 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:27.163034905+01:00" level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" id=e430fed53a6289745b8e88b5a6304969ddf8feb6619d1018a2c59df766f0c66b
Jan 16 06:55:27 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:27.163519219+01:00" level=error msg="Create container failed with error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 06:55:27 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:27.249115352+01:00" level=error msg="Handler for POST /containers/e430fed53a62/start returned error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 06:55:39 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:39.358120742+01:00" level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" id=e430fed53a6289745b8e88b5a6304969ddf8feb6619d1018a2c59df766f0c66b
Jan 16 06:55:39 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:39.358937646+01:00" level=error msg="Create container failed with error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 06:55:39 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T06:55:39.433116316+01:00" level=error msg="Handler for POST /containers/e430fed53a62/start returned error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/c15e5203cb1f71c0369101f144ca958e5e68ec3bc1f480fadd8b916d8ae8c4db/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 09:00:05 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T09:00:05.886477488+01:00" level=warning msg="failed to close stdin: rpc error: code = 2 desc = containerd: container not found"
Jan 16 09:00:06 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T09:00:06+01:00" level=info msg="Firewalld running: false"
Jan 16 20:55:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:55:30.491772452+01:00" level=error msg="failed to close stdin: rpc error: code = 2 desc = containerd: process not found for container"
Jan 16 20:55:46 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:55:46.680475395+01:00" level=error msg="failed to close stdin: rpc error: code = 2 desc = containerd: process not found for container"
Jan 16 20:57:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:30.054825247+01:00" level=error msg="failed to close stdin: rpc error: code = 2 desc = containerd: process not found for container"
Jan 16 20:57:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:30.091050076+01:00" level=warning msg="containerd: unable to save 92f1ad779681ca280177d96a5987db1b19034d5de3043072024276289e7d4619:205a1d25dcf12921be76b90792585559436eca185f816068409ec5d1ecfe5a3a starttime: open /proc/10599/stat: no such file or directory"
Jan 16 20:57:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:30.091232522+01:00" level=info msg="containerd: 92f1ad779681ca280177d96a5987db1b19034d5de3043072024276289e7d4619:205a1d25dcf12921be76b90792585559436eca185f816068409ec5d1ecfe5a3a (pid 10599) has become an orphan, killing it"
Jan 16 20:57:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:30.091771417+01:00" level=error msg="failed to close stdin: rpc error: code = 2 desc = containerd: process not found for container"
Jan 16 20:57:30 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:30.093922806+01:00" level=error msg="Handler for GET /containers/null/json returned error: No such container: null"
Jan 16 20:57:33 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:57:33.185077588+01:00" level=error msg="Handler for GET /containers/undefined_container_00/json returned error: No such container: undefined_container_00"
Jan 16 20:59:40 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:59:40.85405659+01:00" level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n" id=68449baec6e4151c208202d0de1d6f6526508ecd549e11543331c2eeb4783e7f
Jan 16 20:59:40 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:59:40.855153061+01:00" level=error msg="Create container failed with error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 16 20:59:40 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-16T20:59:40.937223071+01:00" level=error msg="Handler for POST /containers/68449baec6e4/start returned error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/28682ac46b11d703f971fe987264356d073573ffbca3f5763222688b17d7cc5a/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\""
Jan 17 09:00:05 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-17T09:00:05.312839014+01:00" level=warning msg="failed to close stdin: rpc error: code = 2 desc = containerd: container not found"
Jan 17 09:00:05 ns352682.ip-91-121-85.eu dockerd[639]: time="2017-01-17T09:00:05+01:00" level=info msg="Firewalld running: false```

DaneEveritt commented 7 years ago

Appears issue is being caused by /etc/timezone being an empty folder rather than a file.

GT500org commented 7 years ago

Shouldn't it be using /etc/localtime instead of /etc/timezone? At least, on CentOS 6 and 7, that is where the system timezone is specified. centos_7_localtime

Anyway, I'm seeing the following in my daemon error.log:

{"name":"wings","hostname":"D5508.dallas-idc.com","pid":10806,"server":"cbd6d6fe-78f1-434b-9c02-3751dfce8683","level":50,"err":{"message":"(HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" ","name":"Error","stack":"Error: (HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" \n    at /srv/daemon/node_modules/docker-modem/lib/modem.js:229:17\n    at getCause (/srv/daemon/node_modules/docker-modem/lib/modem.js:259:7)\n    at Modem.buildPayload (/srv/daemon/node_modules/docker-modem/lib/modem.js:228:5)\n    at IncomingMessage.<anonymous> (/srv/daemon/node_modules/docker-modem/lib/modem.js:204:14)\n    at emitNone (events.js:91:20)\n    at IncomingMessage.emit (events.js:185:7)\n    at endReadableNT (_stream_readable.js:974:12)\n    at _combinedTickCallback (internal/process/next_tick.js:74:11)\n    at process._tickDomainCallback (internal/process/next_tick.js:122:9)"},"msg":"(HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" ","time":"2017-01-17T05:44:25.320Z","v":0}
{"name":"wings","hostname":"D5508.dallas-idc.com","pid":10806,"level":50,"err":{"message":"(HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" ","name":"Error","stack":"Error: (HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" \n    at /srv/daemon/node_modules/docker-modem/lib/modem.js:229:17\n    at getCause (/srv/daemon/node_modules/docker-modem/lib/modem.js:259:7)\n    at Modem.buildPayload (/srv/daemon/node_modules/docker-modem/lib/modem.js:228:5)\n    at IncomingMessage.<anonymous> (/srv/daemon/node_modules/docker-modem/lib/modem.js:204:14)\n    at emitNone (events.js:91:20)\n    at IncomingMessage.emit (events.js:185:7)\n    at endReadableNT (_stream_readable.js:974:12)\n    at _combinedTickCallback (internal/process/next_tick.js:74:11)\n    at process._tickDomainCallback (internal/process/next_tick.js:122:9)"},"msg":"(HTTP code 500) server error - invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"rootfs_linux.go:53: mounting \\\\\\\\\\\\\\\"/etc/timezone\\\\\\\\\\\\\\\" to rootfs \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\\\\\\\\\" at \\\\\\\\\\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\\\\\\\\\" caused \\\\\\\\\\\\\\\"not a directory\\\\\\\\\\\\\\\"\\\\\\\"\\\"\\n\" ","time":"2017-01-17T05:44:25.321Z","v":0}

OS is CentOS 7 (image created by Limestone Networks, using their own internal CentOS repos for official packages)
Kernel is 3.10.0-514.2.2.el7.x86_64
Node.js version is 6.9.4 downloaded from here:
- https://nodejs.org/dist/v6.9.4/node-v6.9.4-linux-x64.tar.xz

Docker packages that are installed (yum list installed | grep docker), using official Docker repo for CentOS:

docker-engine.x86_64                  1.12.6-1.el7.centos            @dockerrepo
docker-engine-selinux.noarch          1.12.6-1.el7.centos            @dockerrepo

bhooten commented 7 years ago

As shared with me by @DaneEveritt, you could either edit this line in (daemon directory)/src/controllers/docker.js to use the proper location of the directory. https://github.com/Pterodactyl/Daemon/blob/develop/src/controllers/docker.js#L375

Or you could delete the directory at /etc/timezone (it's empty) and replace it with a new text file containing your timezone. (chmod 644)

A temporary fix, assuming this is fixed in the next release.

@GT500org

GT500org commented 7 years ago

Even after replacing /etc/timezone with /etc/localtime in Docker.js I'm still getting this error every time I try to start the ARK server:

(HTTP code 500) server error - invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:53: mounting \\\\\\\"/etc/timezone\\\\\\\" to rootfs \\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs\\\\\\\" at \\\\\\\"/var/lib/docker/devicemapper/mnt/95a209561292c56522e221e210ffc274a10411c1e27b448c601e5c2efff8ce7b/rootfs/etc/timezone\\\\\\\" caused \\\\\\\"not a directory\\\\\\\"\\\"\"\n"

I can't find any other files that this is defined in. Is it possible for Node.js or Docker to have cached an old copy of the script somewhere?

I even tried rebooting the server, and I still get the same error.

GT500org commented 7 years ago

Ah, you have to rebuild the container. Now it's working. :)

DaneEveritt commented 7 years ago

Alright, so this issue is confirmed as being CentOS based and we need to do some directory checking to determine wether or not to use /etc/timezone or /etc/localtime

jeppehw13 commented 7 years ago

I have the same problem and it somewhat fixed it but when I try to rebuild the container it says:

/start.sh: line 8: [: /home/container: integer expression expected

curl -sSL -o steamcmd.tar.gz http://media.steampowered.com/installer/steamcmd_linux.tar.gz curl: (6) Could not resolve host: media.steampowered.com
set +x tar (child): steamcmd.tar.gz: Cannot open: No such file or directory tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now Installing requested game, this could take a long time depending on game size and network.
./steamcmd.sh +login anonymous +force_install_dir /home/container +app_update 232250 +quit /start.sh: line 30: ./steamcmd.sh: No such file or directory
set +x cp: cannot stat 'steamcmd/linux32/steamclient.so': No such file or directory ./srcds_run -game tf -console -port 27000 +map cp_dustbowl -strictportbind -norestart /start.sh: line 49: ./srcds_run: No such file or directory (Daemon) Server detected as stopped. (Daemon) Server detected as crashed! Attempting to reboot server now.

don't know if I need to make a new or just post it here. sorry in advance

DaneEveritt commented 7 years ago

That rebuild error looks like our lack of error checking in the script. Your issue is that media.steampowered.com cannot be resolved, therefore nothing is downloaded, but the script tries to continue.

jeppehw13 commented 7 years ago

that much I knew but didn't know why because when I used the curl command on the machine it worked but found out it was just our firewall that blocked the docker ip.

But thanks for your help didn't expect such quick an answer.

DaneEveritt commented 7 years ago

closed by https://github.com/Pterodactyl/Daemon/commit/0160528e517493fb095ad3dfbc0ba898c8c40e48

pterodactyl / panel

[oci runtime error ] 500 Server Error upon Creation of Source Server #253