Key authentication fails for default SSH repository when using multiple repositories with pattern matching

[evpaassen]

When using Pattern Matching and Multiple Repositories, the server doesn't seem to complete authentication with the SSH server, resulting in the repository not being found.

I have the following configuration:

spring:
  cloud:
    config:
      server:
        git:
          uri: git@github.com:evpaassen/scc-config-a.git
          ignore-local-ssh-settings: true
          private-key: |
            -----BEGIN RSA PRIVATE KEY-----
            MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
            (...)
            KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
            -----END RSA PRIVATE KEY-----
          repos:
            repo-b:
              pattern: b/*
              uri: git@github.com:evpaassen/scc-config-b.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJKQIBAAKCAgEA1MyJU9FV3f0UqiAcax88VVZMkE6APbJ0XJParx3RtjpLlE1h
                (...)
                QRA+zPJMseOp2u9ch2eiODzpKyzuqmHpNwpivi8XZZIftF5no8sOIRh8po8a
                -----END RSA PRIVATE KEY-----

Repository scc-config-b connects fine, while the default one, scc-config-a does not. This exception shows up in the log:

2020-09-02 10:43:44.072  WARN 17812 --- [-192.168.99.119] .c.s.e.MultipleJGitEnvironmentRepository : Error occured cloning to base directory.

org.eclipse.jgit.api.errors.InvalidRemoteException: Invalid remote: origin
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:251) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.cloneToBasedir(JGitEnvironmentRepository.java:589) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.copyRepository(JGitEnvironmentRepository.java:564) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.createGitClient(JGitEnvironmentRepository.java:547) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.refresh(JGitEnvironmentRepository.java:268) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.getLocations(JGitEnvironmentRepository.java:246) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository.getLocations(MultipleJGitEnvironmentRepository.java:146) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.AbstractScmEnvironmentRepository.findOne(AbstractScmEnvironmentRepository.java:57) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository.findOne(MultipleJGitEnvironmentRepository.java:187) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.environment.CompositeEnvironmentRepository.findOne(CompositeEnvironmentRepository.java:58) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.cloud.config.server.config.ConfigServerHealthIndicator.doHealthCheck(ConfigServerHealthIndicator.java:73) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
    at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:71) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:39) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:99) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateHealth(HealthEndpointSupport.java:110) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:96) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:74) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:61) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:65) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:55) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:77) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:121) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:96) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
    at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809) ~[na:na]
    at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) ~[na:na]
    at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1466) ~[na:na]
    at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307) ~[na:na]
    at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399) ~[na:na]
    at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:827) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359) ~[na:na]
    at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) ~[na:na]
    at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) ~[na:na]
    at java.base/java.security.AccessController.doPrivileged(Native Method) ~[na:na]
    at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) ~[na:na]
    at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562) ~[na:na]
    at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796) ~[na:na]
    at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677) ~[na:na]
    at java.base/java.security.AccessController.doPrivileged(Native Method) ~[na:na]
    at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.eclipse.jgit.errors.NoRemoteRepositoryException: git@github.com:evpaassen/scc-config-a.git: ERROR: Repository not found.

    at org.eclipse.jgit.transport.TransportGitSsh.cleanNotFound(TransportGitSsh.java:218) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:306) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:170) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:137) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:123) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1271) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:243) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
    ... 55 common frames omitted

Observations

• When using only one repository, it works fine.
• When I set both config repositories to public on Github, it works fine, because they don't need key authentication.
• When I change the configuration to the following, it works fine, even for the default repo (but that doesn't make sense to me, because SCC requires me to configure the uri for the default repository): This is actually not true.

  spring:
  cloud:
    config:
      server:
        git:
          uri: git@github.com:evpaassen/scc-config-a.git
          ignore-local-ssh-settings: true
          private-key: |
            -----BEGIN RSA PRIVATE KEY-----
            MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
            (...)
            KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
            -----END RSA PRIVATE KEY-----
          repos:
            repo-b:
              pattern: b/*
              uri: git@github.com:evpaassen/scc-config-b.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJKQIBAAKCAgEA1MyJU9FV3f0UqiAcax88VVZMkE6APbJ0XJParx3RtjpLlE1h
                (...)
                QRA+zPJMseOp2u9ch2eiODzpKyzuqmHpNwpivi8XZZIftF5no8sOIRh8po8a
                -----END RSA PRIVATE KEY-----
            repo-a:
              pattern: a/*
              uri: git@github.com:evpaassen/scc-config-a.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
                (...)
                KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
                -----END RSA PRIVATE KEY-----

  Sample project I have created a sample project which demonstrates the issue: https://github.com/evpaassen/scc-server. The application connects to to two private repositories, for which I included the deploy keys in the project. The issue should be easily reproducable by just running the ConfigServerApplication.

[spencergibb]

Sorry for the delay. Looking at this, it seems that ssh authentication is by host only right now. I don't see anthing in the parameters of PropertyBasedSshSessionFactory.createSession() that would allow us to choose by repo. There may be a way to plugin via TransportConfigCallback but I'm not sure. Maybe @dyroberts or @ojhughes might know.