search

hasura / gitkube

Build and deploy docker images to Kubernetes using git push
Apache License 2.0
3.81k stars 207 forks source link

Ensure BUILD_ROOT empty before actual check out during pre-receive #106

Open duyleekun opened 5 years ago

duyleekun commented 5 years ago

For some reason, while executing pre-receive, my BUILD_ROOT dir isn't empty. My guess is that the hook doesn't always execute at the end rm -rf $BUILD_ROOT and the git archive $DEPLOY_NEWREV | tar -x -C $BUILD_ROOT doesn't perform any clean up.

tirumaraiselvan commented 5 years ago

Does this happen for each push? Does this happen even if you delete the gitkubed pod from the kube-system namespace (hence creating a new pod) ?

duyleekun commented 5 years ago

Once it's not clean, subsequent failed push won't clear the file (it's failed because of that not deleted file)

I resolve it by exec into the gitkube pod directly and rm the directory. Next time, I will delete the pod instead

Although It's not directly related to this issue, but my guess about failure reason to rm -rf $BUILD_ROOT is following error I got from time to time. Retry the push resolve the error.

remote: Updating Kubernetes deployment: appworker
remote: deployment "appworker" image updated
remote: Waiting for rollout to finish: 1 old replicas are pending termination...
remote: 
remote: 
remote: Updating deployment failed: mofferworker 
remote:   $ Check deployment logs