search

deis / dockerbuilder

Deis dockerbuilder builds your app from a Dockerfile inside a Kubernetes pod, used by Deis Workflow.
https://deis.com
MIT License
17 stars 27 forks source link

Disable buffering for docker output #93

Closed robholland closed 7 years ago

robholland commented 8 years ago

Please disable buffering of the output during a docker build. It can appear as though deploys are hung because output is buffered, for things like bundle install this can be quite frustrating as there isn't much output so the buffer takes a while to fill up.

robholland commented 8 years ago

I also believe this probably responsible for:

Installing proxies 0.2.1 fatal: The remote end hung up unexpectedly fatal: The remote end hung up unexpectedly

during deploys. This happens quite frequently on deploys where a Gemfile has changed.

kmala commented 8 years ago

the remote end hung up errors are mostly due to https://deis.com/docs/workflow/managing-workflow/configuring-load-balancers/#idle-connection-timeouts

robholland commented 8 years ago

We have a 20 minute timeout, so that isn't happening here.

bacongobbler commented 8 years ago

@robholland if you remove the DEIS_ROUTER_SERVICE_HOST/PORT envvars from the builder chart (bypassing the proxy) and assuming you have --insecure-registry set up like with previous releases, do you notice a difference? I've noticed that there has been a lot of timeout reports on Slack, even with the ELB timeout set to 20 minutes.

robholland commented 8 years ago

Will try on Monday and let you know, cheers.

bacongobbler commented 8 years ago

Reason I say try that first is that the builder has been buffering output since inception. Therefore since this is a recent development I'd like to try and see if the proxy_read_timeout or something else is to blame here.

bacongobbler commented 8 years ago

ping @robholland, were you able to try my suggestion last week?

robholland commented 8 years ago

I'm afraid I didn't, no. We're tearing down the staging cluster where we saw this issue, once our new cluster is built I'll come back to this if we still see the issue. I'll close this for now.

bacongobbler commented 8 years ago

@rimusz any reason you're re-opening this ticket?

rimusz commented 8 years ago

it looks like there is an issue again, need to reproduce it on Workflow v2.6

bacongobbler commented 8 years ago

ping @rimusz are you able to reliably reproduce this issue? If so a test case would be useful to identify the issue.

bacongobbler commented 7 years ago

closing due to inactivity. Please re-open when you can reproduce.