rackerlabs / docs-dedicated-networking

Developer documentation source and builds for Automation Integration networking offerings for dedicated customers
https://developer.rackspace.com/docs/dedicated-load-balancers/v2/
3 stars 6 forks source link

Fix Build Repo #12

Closed egonzalez777 closed 8 years ago

egonzalez777 commented 8 years ago

Currently, Strider is failing to build.

Thanks

meker12 commented 8 years ago

Strider builds when you push to master, but it won't build on the PR for some reason.

This is the test results for PR build:

Deconst content directory: /workspace/data/rackerlabs-docs-dedicated-networking-master/job-578535cdc97e9201007965a7/load-balancer/api-docs
Using inferred preparer: quay.io/deconst/preparer-sphinx
Oh no: { [Error: HTTP code is 500 which indicates error: server error - Cannot start container 05365c9e5b8a4c3d3472a0043930684914eae99f501f503f372a2a3ecc3e9a35: [8] System error: read parent: connection reset by peer
]
  reason: 'server error',
  statusCode: 500,
  json: 'Cannot start container 05365c9e5b8a4c3d3472a0043930684914eae99f501f503f372a2a3ecc3e9a35: [8] System error: read parent: connection reset by peer\n' }

@smashwilson Wondering if this job was added before we had PR builder, and automated way to add Strider builds to repos. Would it be useful to try deleting this project from the content-repositories.json file and re-adding it?

smashwilson commented 8 years ago

No:

  1. It's attempting to launch the correct preparer container. That's a Docker error that's occurring during the build.
  2. Removing repositories from content-repositories.json doesn't remove the Strider build, so deleting the project from there wouldn't help. I'd need to delete the repository myself, then re-run the nexus-control build manually to recreate it.

This is a problem with the Docker daemon on the build host. I haven't seen that one before, so I'm not sure what could be causing it. I'll see if there's anything in the logs.

smashwilson commented 8 years ago

From the Docker logs:

Jul 18 17:31:51 deconst-drc-build-blue-001 dockerd[6432]: time="2016-07-18T17:31:51.529465706Z" level=info msg="POST /containers/b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615/start"
Jul 18 17:31:51 deconst-drc-build-blue-001 dockerd[6432]: time="2016-07-18T17:31:51.790812449Z" level=warning msg="signal: killed"
Jul 18 17:31:51 deconst-drc-build-blue-001 dockerd[6432]: time="2016-07-18T17:31:51.897813570Z" level=warning msg="failed to cleanup ipc mounts:\nfailed to umount /var/lib/docker/containers/b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615/shm: invalid argument\nfailed to umount /var/lib/docker/containers/b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615/mqueue: invalid argument"
Jul 18 17:31:51 deconst-drc-build-blue-001 dockerd[6432]: time="2016-07-18T17:31:51.898072312Z" level=error msg="Handler for POST /containers/b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615/start returned error: Cannot start container b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615: [8] System error: read parent: connection reset by peer"
Jul 18 17:31:51 deconst-drc-build-blue-001 dockerd[6432]: time="2016-07-18T17:31:51.898104105Z" level=error msg="HTTP Error" err="Cannot start container b749a4943e601267eefc28980b65e863170756ddde2c2b77a773bb748a348615: [8] System error: read parent: connection reset by peer" statusCode=500

Let's see what Google has to say about this. Filesystem problems maybe?

smashwilson commented 8 years ago

This looks related: docker/docker#16902

smashwilson commented 8 years ago

Ah, that's just for the warning, this one looks related too: docker/docker#14203

smashwilson commented 8 years ago

Well this is a fun read. Here's the root cause:

https://github.com/docker/docker/issues/14203#issuecomment-174177790

This is a Docker bug, triggered by a command line/environment variable configuration for a container being exactly a certain number of bytes.

I'm forcing a CoreOS update on the build server to see if there's a new Docker daemon to grab that includes the fix.

smashwilson commented 8 years ago

Docker upgraded:

core@deconst-drc-build-blue-001 ~ $ docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   1f8f545
 Built:        
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.4
 Git commit:   1f8f545
 Built:        
 OS/Arch:      linux/amd64

And build fixed.