search

fabric8io / fabric8

fabric8 is an open source microservices platform based on Docker, Kubernetes and Jenkins
http://fabric8.io/
1.77k stars 505 forks source link

fabric8-forge fails to commit changes to gogs as it started before the gogs external IP was created #6132

Open rawlingsj opened 8 years ago

rawlingsj commented 8 years ago

sometimes on GKE the fabric8-forge pod starts before the gogs service has an external IP, this causes issues like..

org.eclipse.jgit.errors.TransportException: http://3686598160225327387:30934/gogsadmin/james.git: cannot open git-upload-pack
    at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:510)
    at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:288)
    at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:136)
    at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
    at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1179)
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:128)
    at org.eclipse.jgit.api.PullCommand.call(PullCommand.java:267)
    at io.fabric8.forge.rest.git.RepositoryResource.doPull(RepositoryResource.java:839)
    at io.fabric8.forge.rest.git.RepositoryResource$15.call(RepositoryResource.java:797)
    at io.fabric8.forge.rest.git.GitLockManager.withLock(GitLockManager.java:45)
    at io.fabric8.forge.rest.git.RepositoryResource.gitOperation(RepositoryResource.java:753)
    at io.fabric8.forge.rest.git.RepositoryResource.gitReadOperation(RepositoryResource.java:739)
    at io.fabric8.forge.rest.git.RepositoryResource.gitReadOperation(RepositoryResource.java:733)
    at io.fabric8.forge.rest.CommandsResource.withUIContext(CommandsResource.java:593)
    at io.fabric8.forge.rest.CommandsResource.withUIContext(CommandsResource.java:559)

Ideally it would be good to prevent the fabric8-forge pod being ready to start until the gogs service has an externally routable address using ingress or a GKE loadbalancer

jstrachan commented 8 years ago

looks like its not getting the correct IP for the host. I think I"ve a fix in fabric8 ready to release which should hopefully fix this

jstrachan commented 8 years ago

mind you this fix will only work if there is node with an IP address we can use to access gogs - at least it works on minikube using a local build

rawlingsj commented 8 years ago

yeah I'm not sure if that will work on GKE as we need the loadbalancer or ingress.

I actually exceeded my external IP quota hence why fabric8-forge is using that duff hostname, if we switch to using ingress this might all just work OOTB and no need add extra checks.

I need to test this ingress controller fix, if it works then we should be able to avoid relying on the GKE loadbalancer.. https://github.com/nginxinc/kubernetes-ingress/issues/34