search

fabric8io / fabric8

fabric8 is an open source microservices platform based on Docker, Kubernetes and Jenkins
http://fabric8.io/
1.76k stars 504 forks source link

import from git fails #6293

Open iscra opened 8 years ago

iscra commented 8 years ago

Created new project, selected "import from git", entered Bitbucker URL (HTTPS). Created new secret with user and password, selected it.

Saving secret selection partly fails - secret is saved after refreshing page, but does not go to next step.

Pipeline selection in the next step does not load.

In both cases JS console shows following errors:

[Forge] Failed to load k8s/api/v1/proxy/namespaces/default/services/fabric8-forge:80/api/forge/command/execute/devops-get-overview?secret=bitbucket&secretNamespace=user-secrets-source-admin&kubeUserName=admin <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /api/forge/command/execute/devops-get-overview. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>org.apache.cxf.interceptor.Fault: https://bitbucket.int.stayfriends.de/scm/sf/activity-stream.git: cannot open git-upload-pack
    at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
...
rawlingsj commented 8 years ago

I just tried recreating this and I get a slightly different issue https://bitbucket.org/rawlingsj/fabric8-test-project4.git: not authorized

libs.js:69405 POST http://fabric8.default.beast.fabric8.io/k8s/api/v1/proxy/namespaces/default…-get-overview?secretNamespace=user-secrets-source-admin&kubeUserName=admin 500 (Internal Server Error)(anonymous function) @ libs.js:69405m @ libs.js:69170l @ libs.js:68965o @ libs.js:73777(anonymous function) @ libs.js:73793$eval @ libs.js:75076$digest @ libs.js:74889$apply @ libs.js:75184(anonymous function) @ libs.js:77015a @ libs.js:63358(anonymous function) @ libs.js:63637
libs.js:120926 [Forge] Failed to load k8s/api/v1/proxy/namespaces/default/services/fabric8-forge:80/api/forge/command/execute/devops-get-overview?secretNamespace=user-secrets-source-admin&kubeUserName=admin <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /api/forge/command/execute/devops-get-overview. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.RuntimeException: Failed to command remote repo https://bitbucket.org/rawlingsj/fabric8-test-project4.git due: https://bitbucket.org/rawlingsj/fabric8-test-project4.git: not authorized

Looking at the fabric8-forge pod logs it shows it's using the gogsadmin user to try and clone and not the bitbucket secret I setup with my username/password. The console has the correct default bitbucket secret in local storage

screenshot 2016-08-30 10 09 09

Still investigating and will try to recreate the original issue cannot open git-upload-pack - BTW that normally points to an issue with the URL, is that definitely correct?

iscra commented 8 years ago

Just tried again. I am also getting sometimes the 500 / ont authorized error. Not sure when, seems to depend on which exactly page, dashboard or settings and maybe on state of the project... Anyway looks like the primary issue is indeed wrong secret/user used for external git.

rawlingsj commented 8 years ago

ok thanks for updating, I'm just recreating locally and will look to fix

iscra commented 8 years ago

It seems, that Fabric8 is using wrong authorization, probably the default one (mentioned I think in some other issue as workaround). Using other git than GOGS is really an important feature. Any progress on this or hint how else work with external Git?

iscra commented 8 years ago

When importing from Git using a secret with SSH keys, the following 500 error occurs: ''' [Forge] Failed to load http://fabric8.default.test-kublet-cluster.nbg1.stayfriends.de/k8s/api/v1/p…-get-overview?secretNamespace=user-secrets-source-admin&kubeUserName=admin

Error 500 Server Error

HTTP ERROR 500

Problem accessing /api/forge/command/execute/devops-get-overview. Reason: 

    Server Error

Caused by:

java.lang.RuntimeException: Failed to command remote repo ssh://git@bitbucket.int.stayfriends.de:2222/sf/corporate-website.git due: ssh://git@bitbucket.int.stayfriends.de:2222: org.eclipse.jgit.transport.CredentialItem$YesNoType:The authenticity of host 'bitbucket.int.stayfriends.de' can't be established.
RSA key fingerprint is d8:82:7f:07:b3:6e:79:bf:44:03:45:bc:cf:0b:db:94.
Are you sure you want to continue connecting?
    at io.fabric8.forge.rest.main.ProjectFileSystem.cloneRepo(ProjectFileSystem.java:207)
    at io.fabric8.forge.rest.main.ProjectFileSystem.cloneRepoIfNotExist(ProjectFileSystem.java:188)
    at io.fabric8.forge.rest.git.RepositoryResource$15.call(RepositoryResource.java:757)
    at io.fabric8.forge.rest.git.GitLockManager.withLock(GitLockManager.java:45)
    at io.fabric8.forge.rest.git.RepositoryResource.gitOperation(RepositoryResource.java:753)
    at io.fabric8.forge.rest.git.RepositoryResource.gitWriteoperation(RepositoryResource.java:749)
    at io.fabric8.forge.rest.git.RepositoryResource.gitWriteOperation(RepositoryResource.java:743)
    at io.fabric8.forge.rest.CommandsResource.withUIContext(CommandsResource.java:593)
    at io.fabric8.forge.rest.CommandsResource.withUIContext(CommandsResource.java:561)
    at io.fabric8.forge.rest.CommandsResource.executeCommand(CommandsResource.java:275)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
    at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
    at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
    at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
    at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
    at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254)
    at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
    at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
    at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:217)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
    at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:524)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:745)
Powered by Jetty:// 9.3.10.v20160621

500 '''

Brennan-VanderLaan commented 8 years ago

I am getting the same issues with HTTPS and SSH auth against git and I can't proceed, any updates on this?

fatganz commented 8 years ago

Guys, this issue is critical as most of us are starting our fabric8 experience by trying to import our pet projects from Github or Bitbucket.

P.S. I really appreciate what you're doing. Just let us know how to help you.

jstrachan commented 8 years ago

@iscra @Brennan-VanderLaan @fatganz Sorry about this issue! I've managed to reproduce it; looks like a regression. I'm working on a fix & will cut a release ASAP...

jstrachan commented 8 years ago

@iscra @Brennan-VanderLaan @fatganz so this issue turned out to be a regression in kuisp (the web proxy used in kubernetes inside the web console) stripping query parameters - hence we couldn't pass in the secrets :)

I've cut a release of the console which seems to fix it here. If you delete your rc for fabric8 and run gofabric8 deploy again you should be good to go. Or if you want try edit the rc

kubectl edit rc fabric8

(or edit it via the web console, Runtime -> Replicas -> fabric8 -> click Edit button and change the version string in the YAML editor then hit Save)

then change the image version to 2.2.177 and delete the fabric8 pod to recreate it with the latest version.

Let me know if this fixes it for you

fatganz commented 8 years ago

@jstrachan Can confirm that HTTPS Auth is working properly for me

Brennan-VanderLaan commented 8 years ago

Thanks @jstrachan!

jstrachan commented 8 years ago

Thanks @fatganz @Brennan-VanderLaan am gonna mark this as closed. Let us know if you hit any more issues and we can always reopen

iscra commented 8 years ago

Can confirm, that basically works for me. Is there a way to specify a branch?

Settings/Source/Ref does not seem be saved...

yaniveliash commented 8 years ago

Confirm on HTTPS 👍 SSH auth still doesn't work... 👎

[Forge] Failed to load http://192.168.64.5:32692/k8s/api/v1/proxy/namespaces/default/services/fabr…-get-overview?secretNamespace=user-secrets-source-admin&kubeUserName=admin

Error 500 Server Error

HTTP ERROR 500

Problem accessing /api/forge/command/execute/devops-get-overview. Reason: 

    Server Error

Caused by:

java.lang.RuntimeException: Failed to command remote repo git@bitbucket.org:SOME-main/APP.git due: ssh://git@bitbucket.org:22: org.eclipse.jgit.transport.CredentialItem$YesNoType:The authenticity of host 'bitbucket.org' can't be established.
RSA key fingerprint is 97:8c:1b:ff:ff:14:bb:5c:bb:ec:aa:46:46:74:7c:40.
Are you sure you want to continue connecting?
  at io.fabric8.forge.rest.main.ProjectFileSystem.cloneRepo(ProjectFileSystem.java:207)
  at
donnydavis commented 8 years ago

I am getting the same error on openshift with the latest. This is also the case for the internal git

org.apache.cxf.interceptor.Fault: https://github.com/donnydavis/django-ex.git: not authorized at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99) at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254) at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:217) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:524) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)