search

deis / builder

Git server and application builder for Deis Workflow
https://deis.com
MIT License
40 stars 41 forks source link

Builder is uploading slugs to wrong URL in object storage #130

Closed krancour closed 8 years ago

krancour commented 8 years ago

ATM, builder is uploading built slugs to http://<object storage>/git/home/<app>:git-<short sha>/tar

But workflow tells the slugrunner to get the slug from http://<object storage>/git/home/<app>:git-<long sha>/push/slug.tgz slug.tgz

For example, a slug gets pushed to here:

http://10.3.0.228:9000/git/home/scenic-waterbed:git-afb7db0a/tar

And pulled from here:

http://10.3.0.228:9000/git/home/scenic-waterbed:git-afb7db0a08510b74abbfc52a9e9745971190a37c/push/slug.tgz

cc @arschles

EDIT: Apparently the former URL isn't actually for the slug, but for tarred source. If that's the case, then the slug is actually completely missing from object storage after the build.

bacongobbler commented 8 years ago

We really need to coordinate the path format across projects if possible; this has been the second time IIRC

arschles commented 8 years ago

I agree. I am working on a fix for this specific bug, that adds some unit tests in builder, but the root cause still exists - the paths shouldn't need to be coordinated IMO. Why not just have builder tell workflow where the slug exists, and then workflow just pass that information onto the slugrunner that it launches?

cc/ @helgi

krancour commented 8 years ago

This is what the slugbuilder gets for args:

Environment Variables:
      TAR_URL:      http://10.3.0.228:9000/git/home/august-espresso:git-afb7db0a/tar
      put_url:      http://10.3.0.228:9000/home/august-espresso:git-afb7db0a08510b74abbfc52a9e9745971190a37c/push
      BUILDPACK_URL:

The TAR_URL looks like a legit URL where the tarred source can be found. The put_url looks just like where workflow tells slugrunner to get the slug from. So, I think there's not a mismatch in URL format at all here. I think the problem is just that the slug is actually missing.

helgi commented 8 years ago

I don't mind, whatever works. Can it be added to the push / build hooks, as appropriate?

slack commented 8 years ago

Just ran into this.

Slugrunner:

http://10.3.0.36:9000/git/home/pulsar-footwear:git-5450cbcdaaf9afe6fadd219c94ac9c449bd62413/push/slug.tgz

Minio instance:

$ find /home/minio/git
/home/minio/git/home/pulsar-footwear:git-5450cbcd
/home/minio/git/home/pulsar-footwear:git-5450cbcd/tar
slack commented 8 years ago

With builder debug:

Running in debug mode
Running git hook
read [0000000000000000000000000000000000000000,14068fc773e55ced70ea903a8f928e88952e7772,refs/heads/master]
Workflow request /v2/hooks/push (body elided)
Workflow request POST /v2/hooks/config
{"receive_user":"jhansen","receive_repo":"pulsar-footwear"}
got the following config back for app pulsar-footwear: {Owner:jhansen App:pulsar-footwear Values:map[] Memory:map[] CPU:map[] Tags:map[] UUID:fddd5081-542b-4553-983a-c00fc3d535ac Created:2016-01-27 22:42:59 +0000 UTC Updated:2016-01-27 22:42:59 +0000 UTC}
running [git archive --format=tar.gz --output=pulsar-footwear.tar.gz 14068fc773e55ced70ea903a8f928e88952e7772] in directory /home/git/pulsar-footwear.git
running [tar -xzf pulsar-footwear.tar.gz -C /tmp/] in directory /home/git/pulsar-footwear.git
writing builder manifest to /etc/pulsar-footwear:git-14068fc7
create bucket error: BucketAlreadyExists: The requested bucket name is not available.
remote: status code: 409, request id: R95M2T5AYYEPRJM6
-----> Starting build... but first, coffee!
Starting pod slugbuild-pulsar-footwear-14068fc7-d287deb6
running [kubectl --namespace=deis create -f /etc/pulsar-footwear:git-14068fc7]
pod "slugbuild-pulsar-footwear-14068fc7-d287deb6" created
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis get pods -o yaml slugbuild-pulsar-footwear-14068fc7-d287deb6]
running [kubectl --namespace=deis logs -f slugbuild-pulsar-footwear-14068fc7-d287deb6]
-----> Go app detected
-----> Checking Godeps/Godeps.json file.
-----> Installing go1.4.2... done
-----> Running: godep go install -tags heroku ./...
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size is 1.7M
-----> Build complete.
-----> Launching app.
-----> Launching...
Workflow request POST /v2/hooks/build
{"sha":"14068fc773e55ced70ea903a8f928e88952e7772","receive_user":"jhansen","receive_repo":"pulsar-footwear","image":"pulsar-footwear","procfile":{"web":"example-go"},"dockerfile":""}

-----> Done, pulsar-footwear:v5 deployed to Deis

-----> Use 'deis open' to view this application in your browser

-----> To learn more, use 'deis help' or visit http://deis.io

running [git gc] in directory /home/git/pulsar-footwear.git
To ssh://git@deis.copa.k8s.slack.io:2222/pulsar-footwear.git
 * [new branch]      master -> master

Slugrunner:

k:copa example-go [master]$  kubectl --namespace=pulsar-footwear logs pulsar-footwear-v5-web-4pfxy --previous
+ export HOME=/app
+ HOME=/app
+ mkdir -p /app
++ ls -A /app
+ [[ -n '' ]]
+ [[ -n http://10.3.0.36:9000/git/home/pulsar-footwear:git-14068fc773e55ced70ea903a8f928e88952e7772/push/slug.tgz ]]
+ [[ -e /var/run/secrets/object/store/access-key-id ]]
+ [[ -e /var/run/secrets/object/store/access-secret-key ]]
++ cat /var/run/secrets/object/store/access-key-id
+ keyID=8TZRY2JRWMPT6UMXR6I5
++ cat /var/run/secrets/object/store/access-secret-key
+ secretKey=gbstrOvotMMcg2sMfGUhA5a6Et/EI5ALtIHsobYk
++ awk -F/ '{print $1}'
++ echo http://10.3.0.36:9000/git/home/pulsar-footwear:git-14068fc773e55ced70ea903a8f928e88952e7772/push/slug.tgz
+ protocol=http:
++ awk -F/ '{print $3}'
++ echo http://10.3.0.36:9000/git/home/pulsar-footwear:git-14068fc773e55ced70ea903a8f928e88952e7772/push/slug.tgz
+ domain=10.3.0.36:9000
+ mc config host add http://10.3.0.36:9000 8TZRY2JRWMPT6UMXR6I5 gbstrOvotMMcg2sMfGUhA5a6Et/EI5ALtIHsobYk
+ mc --quiet cp http://10.3.0.36:9000/git/home/pulsar-footwear:git-14068fc773e55ced70ea903a8f928e88952e7772/push/slug.tgz slug.tgz

Minio:

k:copa ~ $ kd exec -it deis-minio-0uixp sh
$ find /home/minio
/home/minio
/home/minio/.minio
/home/minio/.minio/config.json
/home/minio/.bashrc
/home/minio/.profile
/home/minio/.bash_logout
/home/minio/git
/home/minio/git/home
/home/minio/git/home/pulsar-footwear:git-14068fc7
/home/minio/git/home/pulsar-footwear:git-14068fc7/tar
/home/minio/git/home/pulsar-footwear:git-e29a2f09
/home/minio/git/home/pulsar-footwear:git-e29a2f09/tar
/home/minio/git/home/pulsar-footwear:git-5450cbcd
/home/minio/git/home/pulsar-footwear:git-5450cbcd/tar
/home/minio/git/home/pulsar-footwear:git-9ec6fcfd
/home/minio/git/home/pulsar-footwear:git-9ec6fcfd/tar
/home/minio/$multiparts-session.json
/home/minio/$buckets.json.old
/home/minio/$buckets.json
arschles commented 8 years ago

@helgi yes, let's add it to the /v2/hooks/build body. https://github.com/deis/workflow/issues/315