Unable to clone Bitbucket.org repository

[ElMoselYEE]

Summary: I persistently receive "unable to find repository" errors and when looking at the logs, fabric8 has modified my original URL and replaced the host bitbucket.org with gogs. I can see the repo has authenticated in my Dashboard, but the build fails, which tells me I'm doing something wrong but am really struggling to figure out what that is.

Environment:

• Mac OS X
• I've created local fabric8 using: minikube start --vm-driver=xhyve --memory=6144 --cpus=1 --kubernetes-version v1.4.5 && gofabric8 start

Steps to Reproduce:

• add a Jenkinsfile found at https://raw.githubusercontent.com/fabric8io/fabric8-jenkinsfile-library/master/maven/CanaryReleaseStageAndApprovePromote/Jenkinsfile into the root of my repo on master. File is unmodified.
• Navigate to default team dashboard and click 'Create App'
• Choose 'Import from Git' when presented with two options: 'Setup Gogs Secret' and 'Import from Git'
• Type in my HTTPS URL: https://bitbucket.org/myteamname/inventory.audit-service.git. The name is filled in automatically. Next I click 'Save Changes' twice. (No matter how many times I've attempted this, the first click never works)
• On the next page I observe "Repository URL" is now http://gogs/myteamname/inventory.audit-service.git, which is not what I typed in originally.
• Creating a secret using user/pass. Click "Save Selection" twice.
• Click Pipeline tab and click Next.

Actual Result:

• I'm navigated to my Dashboard and on that Dashboard I see the message that Build Failed with the following message:

Started by user anonymous

> /usr/bin/git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

> /usr/bin/git config remote.origin.url http://gogs/myteamname/inventory.audit-service.git # timeout=10

Fetching upstream changes from http://gogs/myteamname/inventory.audit-service.git

> /usr/bin/git --version # timeout=10

> /usr/bin/git fetch --tags --progress http://gogs/myteamname/inventory.audit-service.git +refs/heads/*:refs/remotes/origin/*

ERROR: Error fetching remote repo 'origin'

hudson.plugins.git.GitException: Failed to fetch from http://gogs/myteamname/inventory.audit-service.git

at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:797)

at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1051)

at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1082)

at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)

at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:108)

at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:57)

at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:214)

at hudson.model.ResourceController.execute(ResourceController.java:98)

at hudson.model.Executor.run(Executor.java:404)

Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git fetch --tags --progress http://gogs/myteamname/inventory.audit-service.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:

stdout:

stderr: fatal: repository 'http://gogs/myteamname/inventory.audit-service.git/' not found

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1752)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1495)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:64)

at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:315)

at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:795)

... 8 more

ERROR: null

Finished: FAILURE

Expected Result:

• Pipeline is detected and build succeeds.

Relevant Observations:

• If I click the "Source" tab, I can browse my repository, indicating to me that authentication can succeed, but my pipeline is just configured incorrectly.
• Dashboard correctly shows recent commits to my repo
• I've attempted to use the Clone functionality under Forge tab and Git subsection. This gives me the error "org.apache.cxf.interceptor.Fault: https://bitbucket.org/myteamname/inventory.audit-service.git: Authentication is required but no CredentialsProvider has been registered"
• Using HTTPS or SSH seemingly gives the same result. Using SSH keys or password seemingly gives the same result.

[ElMoselYEE]

I'm following the instructions here: https://medium.com/@kurt.stam/jenkins-on-kubernetes-with-private-git-repositories-4e0186a194eb#.j8cbonxdl

And I think I'm close. Now it's failing because:

Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git fetch --tags --progress git@bitbucket.org:dealersocket/inventory.audit-service.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:

stdout:

stderr: Host key verification failed.

So this seems like not an issue with fabric8 just an issue with how I'm using it. At the very least it's a duplicate of https://github.com/fabric8io/fabric8/issues/6646. We can close this issue but any assistance would be appreciated!

[ElMoselYEE]

I'm able to (somewhat) workaround this issue by adding an entry in /var/jenkins_home/.ssh/known_hosts for bitbucket.org directly on the jenkins container:

eval $(minikube docker-env)
docker ps | grep jenkins-docker # record ID e.g. ea04a970d129
cat ~/.ssh/id_rsa | pbcopy
docker exec -it f23af9168b86 bash -c 'echo `pbpaste` > /var/jenkins_home/.ssh/id_rsa'
docker exec -it f23af9168b86 bash -c 'ssh-keyscan -t rsa bitbucket.org > ~/.ssh/known_hosts'

Edit: Looks like this hack doesn't really get me past the Host key verification failed. issue. Now it's the ad-hoc kubernetes container that cannot do the git fetch. Because the container is only started when the build begins, I don't know when I have a chance to manually log into the container and inject my own key before the script fails.

[rawlingsj]

We can add bitbucket.org and other popular git hosting sites to the ssh configure by default. I can look at that tomorrow as i think it should just be a case of adding entries to https://github.com/fabric8io/fabric8-devops/blob/master/jenkins/src/main/fabric8/jenkins-git-ssh-secret.yml

As for the GIT_URL, that should have been replaced with your bitbucket repo after the pipeline was selected from the fabric8 console. For now you should be able to edit the Jenkinsfile in the root of your repo and replace that property. We'll need to take a look at why this didn't happen automatically.

I have to admit I've not tried importing a project from GitHub or bitbucket recently and there have been some upgrades that could affect this. Hopefully it should be straight forward to resolve any issues now that folks are back from festive holidays. Once I'm at my laptop tomorrow I'll take a look.

On 2 Jan 2017, at 17:11, ElMoselYEE notifications@github.com wrote:

I'm able to get passed this issue by adding an entry in .ssh/known_hosts for bitbucket.org:

eval $(minikube docker-env) docker ps | grep jenkins-docker # record ID e.g. ea04a970d129 docker exec -it ea04a970d129 ssh-keyscan -t rsa bitbucket.org > ~/.ssh/known_hosts Now on to debug the next issue:

groovy.lang.MissingPropertyException: No such property: GIT_URL for class: WorkflowScript

Can I ask if I'm doing something non-standard or if everyone experiences issues setting up fabric8? From the videos it looked like it'd be quick to set up but it's honestly been a challenge and I haven't succeeded in anything yet, even setting up example project.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ElMoselYEE]

I've attempted to add my SSH keys to the Secrets jenkins-git-ssh, jenkins-master-ssh, and jenkins-ssh-config, but I don't see any evidence those keys are being injected into the container jenkins-docker container. When I manually copy the same keys it seems to work (on that container only, however).

[h4ckroot]

any update on this issue please ? I am facing the same.

[aleksarias]

I am also facing this problem on version 2.2.311 @unoriginalbanter, you should follow this issue too.

[ElMoselYEE]

@rawlingsj when is the correct place in the process to "add entries to jenkins-git-ssh-secret"? I've been fiddling around with this for a while but still no success, maybe if there was an example of something similar being accomplished?

[oweise]

FWIW, I could workaround this issue by changing the provided Jenkinsfile:

on line: git GIT_URL

to: git credentialsId: 'id-of-the-git-creds', url: '

[seboudry]

Hi!

This workaround is quite dirty, but I've managed to make it work on fabric8 2.4.24. I'm using an external repo (Bitbucket Server) hosted on a private server. I've created a passphrase-less SSH key pair and authorize this identity on a bitbucket project.

1. Jenkins configuration Add a new credential of type SSH with the private key content in plain text [jenkins-url/credentials/store/system/domain/_/newCredentials]()

2. Fabric8 Kubernetes secrets I use "default" team so change your team namespace acordingly. Add both SSH key parts on the secret default/jenkins-git-ssh. Add this SSH config on secret default/jenkins-ssh-config:

   Host <git-repo-host>
   Port <git-repo-port>
   User git
   IdentityFile /root/.ssh-git/ssh-key
   StrictHostKeyChecking no

   Add StrictHostKeyChecking line only if you use a self-signed certificate.

3. JenkinsFile of project In mavenNode just before git command add:

   sh 'chmod 600 /root/.ssh-git/ssh-key'

I also took inspiration of this blog post.

[basu76]

We are kind of passing fabric8 for now, purely because of this issue. It was reported few months back :(

[ChadCorsentino]

Has this issue been resolved? I was able to work around it pushing and pulling from the host machine of my minikube instance, directly into the fabric8 gogs repository. However, my team would prefer to pull from Bitbucket in the long haul.

[ElMoselYEE]

I opened the issue but don't currently have Fabric installed to confirm if the above workaround works for me. I'll try this week to swing back around and give it another go.