search

buildkite / sockguard

A proxy for docker.sock that enforces access control and isolated privileges
MIT License
143 stars 22 forks source link

Implement BuildKit + Moby Support (by Docker CE 18.09 release) #29

Open CpuID opened 6 years ago

CpuID commented 6 years ago

When trying to use Docker 18.06 with "experimental": true in /etc/docker/daemon.json, and attempting to build a super basic Dockerfile:

You will hit this:

root@17fd98e809da:/blah# docker build .
ERRO[0000] failed to dial gRPC: unable to upgrade to h2c, received 501 
Sending build context to Docker daemon  2.048kB
context canceled
root@17fd98e809da:/blah# cat Dockerfile 
FROM alpine:3.8

RUN apk add --no-cache bash

CMD [ "ls", "-la" ]
root@17fd98e809da:/blah#

Sockguard logs:

sockguard_1  | #2 09:23:55.130152 POST - /session - 0b
sockguard_1  | #2 09:23:55.130525 Handler returned error "POST /session not implemented yet"
sockguard_1  | #3 09:23:55.130841 POST - /v1.38/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=r0fwk1oznxrhbc3pl0c7le7g0&shmsize=0&target=&ulimits=null&version=1 - -1b
sockguard_1  | #3 09:23:55.130993 Adding label com.buildkite.sockguard.owner=sockguard-pid-1 to querystring: /v1.38/build buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=r0fwk1oznxrhbc3pl0c7le7g0&shmsize=0&target=&ulimits=null&version=1
sockguard_1  | #3 09:23:55.132059 Error copying request to target: unexpected EOF

I'm pretty sure this is partially related to the BuildKit additions. Note: this test didn't have DOCKER_BUILDKIT=1 set, but if you enable that you get other related timeouts.

root@17fd98e809da:/blah# DOCKER_BUILDKIT=1 docker build .
[+] Building 5.0s (2/2) FINISHED                                                                                                                                          
 => ERROR local://dockerfile (Dockerfile)                                                                                                                            5.0s
 => ERROR local://context (.dockerignore)                                                                                                                            5.0s
------
 > local://dockerfile (Dockerfile):
------
------
 > local://context (.dockerignore):
------
failed to dial gRPC: unable to upgrade to h2c, received 501

And the debug logs:

sockguard_1  | #6 09:30:25.714712 POST - /v1.38/build?buildargs=%7B%7D&buildid=e13f1a2d1559d4f7e7f8f5ae1f9a45192feec2e45788e2a741c1fde31d1e3edd&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=%7B%7D&memory=0&memswap=0&networkmode=default&remote=client-session&rm=1&session=o128vvqwyi6nxegba8t0tiw8s&shmsize=0&target=&ulimits=null&version=2 - 0b
sockguard_1  | #6 09:30:25.714936 Adding label com.buildkite.sockguard.owner=sockguard-pid-1 to querystring: /v1.38/build buildargs=%7B%7D&buildid=e13f1a2d1559d4f7e7f8f5ae1f9a45192feec2e45788e2a741c1fde31d1e3edd&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=%7B%7D&memory=0&memswap=0&networkmode=default&remote=client-session&rm=1&session=o128vvqwyi6nxegba8t0tiw8s&shmsize=0&target=&ulimits=null&version=2
sockguard_1  | #6 09:30:25.715308 > POST /v1.38/build?buildargs=%7B%7D&buildid=e13f1a2d1559d4f7e7f8f5ae1f9a45192feec2e45788e2a741c1fde31d1e3edd&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=&labels=%7B%22com.buildkite.sockguard.owner%22%3A%22sockguard-pid-1%22%7D&memory=0&memswap=0&networkmode=default&remote=client-session&rm=1&session=o128vvqwyi6nxegba8t0tiw8s&shmsize=0&target=&ulimits=null&version=2 HTTP/1.1
sockguard_1  | #6 09:30:25.715321 > Host: docker
sockguard_1  | #6 09:30:25.715322 > User-Agent: Docker-Client/18.06.1-ce (linux)
sockguard_1  | #6 09:30:25.715324 > Content-Length: 0
sockguard_1  | #6 09:30:25.715325 > Connection: close
sockguard_1  | #6 09:30:25.715326 > Content-Type: application/x-tar
sockguard_1  | #6 09:30:25.715327 > X-Registry-Config: bnVsbA==
sockguard_1  | #6 09:30:25.715329 > 
sockguard_1  | #6 09:30:25.718577 < HTTP/1.1 200 OK
sockguard_1  | #6 09:30:25.718660 < Api-Version: 1.38
sockguard_1  | #6 09:30:25.718663 < Content-Type: application/json
sockguard_1  | #6 09:30:25.718665 < Docker-Experimental: true
sockguard_1  | #6 09:30:25.718667 < Ostype: linux
sockguard_1  | #6 09:30:25.718669 < Server: Docker/18.06.0-ce (linux)
sockguard_1  | #6 09:30:25.718671 < Date: Thu, 23 Aug 2018 09:30:25 GMT
sockguard_1  | #6 09:30:25.718673 < Connection: close
sockguard_1  | #6 09:30:25.718675 < Transfer-Encoding: chunked
sockguard_1  | #6 09:30:25.718680 < 
sockguard_1  | #6 09:30:25.718682 < 16f
sockguard_1  | #6 09:30:25.718688 < {"id":"moby.buildkit.trace","aux":"CngKR3NoYTI1Njo4MDAzM2I5Y2VlNThiMzlhNTc2NWUzNzk0NDY4MzQ1YTNlOWQyYWMyNzU5ZTQxYmU3MTNmNTJhOTcxZTBjNWFiGh9sb2NhbDovL2RvY2tlcmZpbGUgKERvY2tlcmZpbGUpKgwIsf352wUQh7ma1gIKeApHc2hhMjU2OjdiNjBhZjNhM2FlZjlhMTBmNzk2ODRiOGJkODA2NDA4YTg3ZDMyZjMyOGIxZjEzMWZmYmU1ZWQ3NDliNzQxOGMaH2xvY2FsOi8vY29udGV4dCAoLmRvY2tlcmlnbm9yZSkqDAix/fnbBRCr1qjWAg=="}
sockguard_1  | #6 09:30:25.718690 < 
sockguard_1  | #7 09:30:25.722004 POST - /v1.38/build/cancel?id=e13f1a2d1559d4f7e7f8f5ae1f9a45192feec2e45788e2a741c1fde31d1e3edd - 0b
sockguard_1  | #7 09:30:25.723060 Handler returned error "POST /v1.38/build/cancel not implemented yet"
sockguard_1  | #6 09:30:30.723051 < 25f
sockguard_1  | #6 09:30:30.723458 < {"id":"moby.buildkit.trace","aux":"CtIBCkdzaGEyNTY6ODAwMzNiOWNlZTU4YjM5YTU3NjVlMzc5NDQ2ODM0NWEzZTlkMmFjMjc1OWU0MWJlNzEzZjUyYTk3MWUwYzVhYhofbG9jYWw6Ly9kb2NrZXJmaWxlIChEb2NrZXJmaWxlKSoMCLH9+dsFEIe5mtYCMgwItv352wUQn4Hj1wI6Sm5vIGFjdGl2ZSBzZXNzaW9uIGZvciBvMTI4dnZxd3lpNm54ZWdiYTh0MHRpdzhzOiBjb250ZXh0IGRlYWRsaW5lIGV4Y2VlZGVkCtIBCkdzaGEyNTY6N2I2MGFmM2EzYWVmOWExMGY3OTY4NGI4YmQ4MDY0MDhhODdkMzJmMzI4YjFmMTMxZmZiZTVlZDc0OWI3NDE4YxofbG9jYWw6Ly9jb250ZXh0ICguZG9ja2VyaWdub3JlKSoMCLH9+dsFEKvWqNYCMgwItv352wUQhYHj1wI6Sm5vIGFjdGl2ZSBzZXNzaW9uIGZvciBvMTI4dnZxd3lpNm54ZWdiYTh0MHRpdzhzOiBjb250ZXh0IGRlYWRsaW5lIGV4Y2VlZGVk"}
sockguard_1  | #6 09:30:30.723476 < 
sockguard_1  | #6 09:30:30.723517 < bf
sockguard_1  | #6 09:30:30.723566 < {"errorDetail":{"message":"no active session for o128vvqwyi6nxegba8t0tiw8s: context deadline exceeded"},"error":"no active session for o128vvqwyi6nxegba8t0tiw8s: context deadline exceeded"}

Will likely need to implement this before Docker CE 18.09:

Sockguard debug logs (without DOCKER_BUILDKIT=1)

sockguard_1  | #2 09:28:36.153557 POST - /v1.38/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=lyozb0039wtzfd5a76d3avubo&shmsize=0&target=&ulimits=null&version=1 - -1b
sockguard_1  | #2 09:28:36.153667 Adding label com.buildkite.sockguard.owner=sockguard-pid-1 to querystring: /v1.38/build buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=lyozb0039wtzfd5a76d3avubo&shmsize=0&target=&ulimits=null&version=1
sockguard_1  | #2 09:28:36.154020 > POST /v1.38/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%22com.buildkite.sockguard.owner%22%3A%22sockguard-pid-1%22%7D&memory=0&memswap=0&networkmode=default&rm=1&session=lyozb0039wtzfd5a76d3avubo&shmsize=0&target=&ulimits=null&version=1 HTTP/1.1
sockguard_1  | #2 09:28:36.154025 > Host: docker
sockguard_1  | #2 09:28:36.154027 > User-Agent: Docker-Client/18.06.1-ce (linux)
sockguard_1  | #2 09:28:36.154028 > Transfer-Encoding: chunked
sockguard_1  | #2 09:28:36.154030 > Connection: close
sockguard_1  | #2 09:28:36.154031 > Content-Type: application/x-tar
sockguard_1  | #2 09:28:36.154032 > X-Registry-Config: e30=
sockguard_1  | #2 09:28:36.154034 > 
sockguard_1  | #2 09:28:36.154035 > 800
sockguard_1  | #2 09:28:36.154036 > Dockerfile0100644000000000000000000000010213337476450013054 0ustar00rootroot00000000000000FROM alpine:3.8
sockguard_1  | #2 09:28:36.154038 > 
sockguard_1  | #2 09:28:36.154039 > RUN apk add --no-cache bash
sockguard_1  | #2 09:28:36.154040 > 
sockguard_1  | #2 09:28:36.154042 > CMD [ "ls", "-la" ]
sockguard_1  | #2 09:28:36.154046 > 
sockguard_1  | #2 09:28:36.154166 > 0
sockguard_1  | #2 09:28:36.154169 > 
sockguard_1  | #3 09:28:36.155706 POST - /session - 0b
sockguard_1  | #3 09:28:36.155947 Handler returned error "POST /session not implemented yet"
sockguard_1  | #2 09:28:36.156512 Copied 0 bytes from connection
sockguard_1  | #2 09:28:41.246646 Err: write unix /var/run/docker/sockguard.sock->@: write: broken pipe
sockguard_1  | #2 09:28:41.246775 Copied 0 bytes from socket
sockguard_1  | #2 09:28:41.246791 Done, closing
CpuID commented 6 years ago

Sucks, API version 1.38 doesn't have public docs yet, which isn't super helpful :)