cloudfoundry / cf-deployment

The canonical open source deployment manifest for Cloud Foundry
Apache License 2.0
294 stars 306 forks source link

CF SSH is not working on vSphere #177

Closed prashantgnet closed 7 years ago

prashantgnet commented 7 years ago

"cf ssh app_name" gives following error message on vSphere environment.

Error opening SSH connection: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

Kindly let me know in case of any additional details are required.

Thanks, Prashant

cf-gitbot commented 7 years ago

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/148738691

The labels on this github issue will be updated when the story is started.

prashantgnet commented 7 years ago

Output of "cf ssh" after enabling CF_TRACE

[centos@bastion ~]$ cf ssh cf-spring

REQUEST: [2017-07-13T01:42:40-04:00]
GET /v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/apps?q=name%3Acf-spring&inline-relations-depth=1 HTTP/1.1
Host: api.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 401 Unauthorized
Connection: close
Content-Length: 97
Content-Type: application/json;charset=utf-8
Date: Thu, 13 Jul 2017 05:52:45 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: f0ce5c56-f580-461c-5bf7-3de374cb1a30
X-Vcap-Request-Id: 051f9487-1956-403c-7813-cb6c8cb1eeed::6d7e94e2-1dd6-430f-9d69-5aee8b5077b1

{
  "description": "Invalid Auth Token",
  "error_code": "CF-InvalidAuthToken",
  "code": 1000
}

REQUEST: [2017-07-13T01:42:40-04:00]
POST /oauth/token HTTP/1.1
Host: login.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

grant_type=refresh_token&refresh_token=eyJhbGciOiJSUzI1NiIsImtpZCI6ImtleS0xIiwidHlwIjoiSldUIn0.eyJqdGkiOiI4MDhlOGIwMTY3ODA0ZDNlYjg3MTM2MmU4OWI2MTUwNi1yIiwic3ViIjoiMmEyYWEwOTctYzBmNi00ZTJhLTgxMmItNWRjMzI0ZTRkZTBiIiwic2NvcGUiOlsib3BlbmlkIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzLndyaXRlIiwic2NpbS5yZWFkIiwiY2xvdWRfY29udHJvbGxlci5hZG1pbiIsInVhYS51c2VyIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzLnJlYWQiLCJjbG91ZF9jb250cm9sbGVyLnJlYWQiLCJwYXNzd29yZC53cml0ZSIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJuZXR3b3JrLmFkbWluIiwiZG9wcGxlci5maXJlaG9zZSIsInNjaW0ud3JpdGUiXSwiaWF0IjoxNDk5ODU5ODI1LCJleHAiOjE1MDI0NTE4MjUsImNpZCI6ImNmIiwiY2xpZW50X2lkIjoiY2YiLCJpc3MiOiJodHRwczovL3VhYS5wYXNzMjNydW50aW1lLnBhc3Muc2FzLmNvbS9vYXV0aC90b2tlbiIsInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfbmFtZSI6ImFkbWluIiwib3JpZ2luIjoidWFhIiwidXNlcl9pZCI6IjJhMmFhMDk3LWMwZjYtNGUyYS04MTJiLTVkYzMyNGU0ZGUwYiIsInJldl9zaWciOiI4ZjEwZWI1ZCIsImF1ZCI6WyJzY2ltIiwiY2xvdWRfY29udHJvbGxlciIsInBhc3N3b3JkIiwiY2YiLCJ1YWEiLCJvcGVuaWQiLCJkb3BwbGVyIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzIiwibmV0d29yayJdfQ.DSHaxc6yxhOjEvcvTzaxLOw8ef0UE3A1JzCmYbpeQSx_rt-wNivsr6FIlwYw7yDmlje5gkekUEOfsADUEARY6oFffehUm7xfRzgurxfzIlUkIFajEvAIyjVl-yOtSbTrgnugmSIfJ3Q49GEr7CnbTgH5oAb5EVeKX8ihqBlPX1iiewyOKtNT21QZrVd4WcjDATP2IXQrRkc6bQzSSrp1cV3AyFJM1NsclTouiIhQfx2ZHJuGm5C4pIQ8pYVMv__4agRuNSqfx2ZKUSZ-aBmy8B1EfJlV1xQVN86LXIn88_HVw2jriNq2zR_qznyaezhg_7DqfIFOJP9ZEi0ROS-7oQ&scope=

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json;charset=UTF-8
Date: Thu, 13 Jul 2017 05:52:45 GMT
Expires: 0
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 70bede52-861b-4681-7d61-91eb46f13795
X-Xss-Protection: 1; mode=block

bd9
{"access_token":"[PRIVATE DATA HIDDEN]","token_type":"[PRIVATE DATA HIDDEN]","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":599,"scope":"openid routing.router_groups.write scim.read cloud_controller.admin uaa.user routing.router_groups.read cloud_controller.read password.write cloud_controller.write network.admin doppler.firehose scim.write","jti":"45407999938f4c038e89fa990beff0ad"}
0

REQUEST: [2017-07-13T01:42:40-04:00]
GET /v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/apps?q=name%3Acf-spring&inline-relations-depth=1 HTTP/1.1
Host: api.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 200 OK
Connection: close
Content-Length: 6229
Content-Type: application/json;charset=utf-8
Date: Thu, 13 Jul 2017 05:52:46 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: c99f5f27-ed10-4a7a-6d90-693593258a05
X-Vcap-Request-Id: 32e6ef74-7629-4747-433b-1e0e5dd80d04::428a875f-6d50-4894-b4f9-b8b390601f85

{
  "total_results": 1,
  "total_pages": 1,
  "prev_url": null,
  "next_url": null,
  "resources": [
    {
      "metadata": {
        "guid": "12d8da0a-42a2-4090-a184-a3d3087a3db0",
        "url": "/v2/apps/12d8da0a-42a2-4090-a184-a3d3087a3db0",
        "created_at": "2017-07-12T11:44:11Z",
        "updated_at": "2017-07-12T11:44:17Z"
      },
      "entity": {
        "name": "cf-spring",
        "production": false,
        "space_guid": "0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
        "stack_guid": "a2552b14-3596-47d4-81b4-5530658d18d7",
        "buildpack": null,
        "detected_buildpack": "container-security-provider=1.6.0_RELEASE java-buildpack=v3.18-https://github.com/cloudfoundry/java-buildpack.git#841ecb2 open-jdk-like-jre=1.8.0_131 open-jdk-like-memory-calculator=2.0.2_RELEASE open-jdk-like-security-providers security-providers spri...",
        "detected_buildpack_guid": "f7eeef1b-8fc0-484b-a7f3-5cdad0b38902",
        "environment_json": {

        },
        "memory": 512,
        "instances": 1,
        "disk_quota": 1024,
        "state": "STARTED",
        "version": "c9ca5838-89b1-4d29-8797-a5185b96c6e7",
        "command": null,
        "console": false,
        "debug": null,
        "staging_task_id": "b4148506-1710-4505-bcc9-3e8169f601ff",
        "package_state": "STAGED",
        "health_check_type": "port",
        "health_check_timeout": null,
        "health_check_http_endpoint": null,
        "staging_failed_reason": null,
        "staging_failed_description": null,
        "diego": true,
        "docker_image": null,
        "docker_credentials": {
          "username": null,
          "password": null
        },
        "package_updated_at": "2017-07-12T11:44:11Z",
        "detected_start_command": "CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-2.0.2_RELEASE -memorySizes=metaspace:64m..,stack:228k.. -memoryWeights=heap:65,metaspace:10,native:15,stack:10 -memoryInitials=heap:100%,metaspace:100% -stackThreads=300 -totMemory=$MEMORY_LIMIT) && SERVER_PORT=$PORT JAVA_HOME=$PWD/.java-buildpack/open_jdk_jre JAVA_OPTS=\"-Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh $CALCULATED_MEMORY -Djava.ext.dirs=$PWD/.java-buildpack/container_security_provider:$PWD/.java-buildpack/open_jdk_jre/lib/ext -Djava.security.properties=$PWD/.java-buildpack/security_providers/java.security\" exec $PWD/.java-buildpack/spring_boot_cli/bin/spring run app.groovy",
        "enable_ssh": true,
        "ports": [
          8080
        ],
        "space_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
        "space": {
          "metadata": {
            "guid": "0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
            "url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
            "created_at": "2017-07-12T11:43:50Z",
            "updated_at": "2017-07-12T11:43:50Z"
          },
          "entity": {
            "name": "test",
            "organization_guid": "b39d6bfd-4b36-46b2-b037-3f216f21593c",
            "space_quota_definition_guid": null,
            "isolation_segment_guid": null,
            "allow_ssh": true,
            "organization_url": "/v2/organizations/b39d6bfd-4b36-46b2-b037-3f216f21593c",
            "developers_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/developers",
            "managers_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/managers",
            "auditors_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/auditors",
            "apps_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/apps",
            "routes_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/routes",
            "domains_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/domains",
            "service_instances_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/service_instances",
            "app_events_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/app_events",
            "events_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/events",
            "security_groups_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/security_groups",
            "staging_security_groups_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886/staging_security_groups"
          }
        },
        "stack_url": "/v2/stacks/a2552b14-3596-47d4-81b4-5530658d18d7",
        "stack": {
          "metadata": {
            "guid": "a2552b14-3596-47d4-81b4-5530658d18d7",
            "url": "/v2/stacks/a2552b14-3596-47d4-81b4-5530658d18d7",
            "created_at": "2017-07-12T11:37:17Z",
            "updated_at": "2017-07-12T11:37:17Z"
          },
          "entity": {
            "name": "cflinuxfs2",
            "description": "Cloud Foundry Linux-based filesystem"
          }
        },
        "routes_url": "/v2/apps/12d8da0a-42a2-4090-a184-a3d3087a3db0/routes",
        "routes": [
          {
            "metadata": {
              "guid": "13e9a96e-78dd-40dc-be29-80b19728bf5f",
              "url": "/v2/routes/13e9a96e-78dd-40dc-be29-80b19728bf5f",
              "created_at": "2017-07-12T11:44:11Z",
              "updated_at": "2017-07-12T11:44:11Z"
            },
            "entity": {
              "host": "cf-spring-picaresque-ratchet",
              "path": "",
              "domain_guid": "b06e9265-2046-4d53-88c3-dbaab629451e",
              "space_guid": "0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
              "service_instance_guid": null,
              "port": null,
              "domain_url": "/v2/shared_domains/b06e9265-2046-4d53-88c3-dbaab629451e",
              "space_url": "/v2/spaces/0626dd5b-8d80-4fe1-be41-e2f3df4ea886",
              "apps_url": "/v2/routes/13e9a96e-78dd-40dc-be29-80b19728bf5f/apps",
              "route_mappings_url": "/v2/routes/13e9a96e-78dd-40dc-be29-80b19728bf5f/route_mappings"
            }
          }
        ],
        "events_url": "/v2/apps/12d8da0a-42a2-4090-a184-a3d3087a3db0/events",
        "service_bindings_url": "/v2/apps/12d8da0a-42a2-4090-a184-a3d3087a3db0/service_bindings",
        "service_bindings": [

        ],
        "route_mappings_url": "/v2/apps/12d8da0a-42a2-4090-a184-a3d3087a3db0/route_mappings"
      }
    }
  ]
}

REQUEST: [2017-07-13T01:42:40-04:00]
GET /v2/info HTTP/1.1
Host: api.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 200 OK
Connection: close
Content-Length: 632
Content-Type: application/json;charset=utf-8
Date: Thu, 13 Jul 2017 05:52:46 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 02e291ad-5082-4556-7009-4830aef1760d
X-Vcap-Request-Id: 75764019-bc60-498b-6f08-f0ae15bde8c2::f1ef170c-bdac-444e-8843-081d72744d68

{"name":"","build":"","support":"","version":0,"description":"","authorization_endpoint":"https://login.runtime.pass.sas.com","token_endpoint":"[PRIVATE DATA HIDDEN]","min_cli_version":null,"min_recommended_cli_version":null,"api_version":"2.85.0","app_ssh_endpoint":"ssh.runtime.pass.sas.com:2222","app_ssh_host_key_fingerprint":"c1:c2:c5:6d:c3:1b:21:10:fe:f2:9c:d3:92:1f:b8:ad","app_ssh_oauth_client":"ssh-proxy","doppler_logging_endpoint":"wss://doppler.runtime.pass.sas.com:443","routing_endpoint":"https://api.runtime.pass.sas.com/routing","user":"2a2aa097-c0f6-4e2a-812b-5dc324e4de0b"}

REQUEST: [2017-07-13T01:42:40-04:00]
GET /v2/info HTTP/1.1
Host: api.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/json
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 200 OK
Connection: close
Content-Length: 632
Content-Type: application/json;charset=utf-8
Date: Thu, 13 Jul 2017 05:52:46 GMT
Server: nginx
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 5f753f0d-a414-4495-4b8b-3cbe84c6be8d
X-Vcap-Request-Id: 612ea7eb-ef16-4188-47f6-752c174bb974::b84cd6db-7872-4658-9f82-e78d96c7df0b

{"name":"","build":"","support":"","version":0,"description":"","authorization_endpoint":"https://login.runtime.pass.sas.com","token_endpoint":"[PRIVATE DATA HIDDEN]","min_cli_version":null,"min_recommended_cli_version":null,"api_version":"2.85.0","app_ssh_endpoint":"ssh.runtime.pass.sas.com:2222","app_ssh_host_key_fingerprint":"c1:c2:c5:6d:c3:1b:21:10:fe:f2:9c:d3:92:1f:b8:ad","app_ssh_oauth_client":"ssh-proxy","doppler_logging_endpoint":"wss://doppler.runtime.pass.sas.com:443","routing_endpoint":"https://api.runtime.pass.sas.com/routing","user":"2a2aa097-c0f6-4e2a-812b-5dc324e4de0b"}

REQUEST: [2017-07-13T01:42:40-04:00]
POST /oauth/token HTTP/1.1
Host: login.runtime.pass.sas.com
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Connection: close
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.28.0+9e024bd.2017-06-27 / linux

grant_type=refresh_token&refresh_token=eyJhbGciOiJSUzI1NiIsImtpZCI6ImtleS0xIiwidHlwIjoiSldUIn0.eyJqdGkiOiI4MDhlOGIwMTY3ODA0ZDNlYjg3MTM2MmU4OWI2MTUwNi1yIiwic3ViIjoiMmEyYWEwOTctYzBmNi00ZTJhLTgxMmItNWRjMzI0ZTRkZTBiIiwic2NvcGUiOlsib3BlbmlkIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzLndyaXRlIiwic2NpbS5yZWFkIiwiY2xvdWRfY29udHJvbGxlci5hZG1pbiIsInVhYS51c2VyIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzLnJlYWQiLCJjbG91ZF9jb250cm9sbGVyLnJlYWQiLCJwYXNzd29yZC53cml0ZSIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJuZXR3b3JrLmFkbWluIiwiZG9wcGxlci5maXJlaG9zZSIsInNjaW0ud3JpdGUiXSwiaWF0IjoxNDk5ODU5ODI1LCJleHAiOjE1MDI0NTE4MjUsImNpZCI6ImNmIiwiY2xpZW50X2lkIjoiY2YiLCJpc3MiOiJodHRwczovL3VhYS5wYXNzMjNydW50aW1lLnBhc3Muc2FzLmNvbS9vYXV0aC90b2tlbiIsInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfbmFtZSI6ImFkbWluIiwib3JpZ2luIjoidWFhIiwidXNlcl9pZCI6IjJhMmFhMDk3LWMwZjYtNGUyYS04MTJiLTVkYzMyNGU0ZGUwYiIsInJldl9zaWciOiI4ZjEwZWI1ZCIsImF1ZCI6WyJzY2ltIiwiY2xvdWRfY29udHJvbGxlciIsInBhc3N3b3JkIiwiY2YiLCJ1YWEiLCJvcGVuaWQiLCJkb3BwbGVyIiwicm91dGluZy5yb3V0ZXJfZ3JvdXBzIiwibmV0d29yayJdfQ.DSHaxc6yxhOjEvcvTzaxLOw8ef0UE3A1JzCmYbpeQSx_rt-wNivsr6FIlwYw7yDmlje5gkekUEOfsADUEARY6oFffehUm7xfRzgurxfzIlUkIFajEvAIyjVl-yOtSbTrgnugmSIfJ3Q49GEr7CnbTgH5oAb5EVeKX8ihqBlPX1iiewyOKtNT21QZrVd4WcjDATP2IXQrRkc6bQzSSrp1cV3AyFJM1NsclTouiIhQfx2ZHJuGm5C4pIQ8pYVMv__4agRuNSqfx2ZKUSZ-aBmy8B1EfJlV1xQVN86LXIn88_HVw2jriNq2zR_qznyaezhg_7DqfIFOJP9ZEi0ROS-7oQ&scope=

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json;charset=UTF-8
Date: Thu, 13 Jul 2017 05:52:45 GMT
Expires: 0
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: d04fbb41-a100-4f76-5136-0415527bc1d3
X-Xss-Protection: 1; mode=block

bd9
{"access_token":"[PRIVATE DATA HIDDEN]","token_type":"[PRIVATE DATA HIDDEN]","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":599,"scope":"openid routing.router_groups.write scim.read cloud_controller.admin uaa.user routing.router_groups.read cloud_controller.read password.write cloud_controller.write network.admin doppler.firehose scim.write","jti":"6128eed266be40e6a512fac71744d9ca"}
0

REQUEST: [2017-07-13T01:42:40-04:00]
GET /login?code=EpUAYBaBGh HTTP/0.0
Host: uaa.runtime.pass.sas.com
Authorization: [PRIVATE DATA HIDDEN]
Referer: https://uaa.runtime.pass.sas.com/oauth/authorize?client_id=ssh-proxy&grant_type=authorization_code&response_type=code

RESPONSE: [2017-07-13T01:42:40-04:00]
HTTP/1.1 302 Found
Connection: close
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Date: Thu, 13 Jul 2017 05:52:45 GMT
Expires: 0
Location: https://uaa.runtime.pass.sas.com/login?code=EpUAYBaBGh
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 4e6e88e6-070c-490b-5656-c0f1765620e4
X-Xss-Protection: 1; mode=block
Content-Length: 0

FAILED
Error opening SSH connection: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
[centos@bastion ~]$ 
dsabeti commented 7 years ago

Hi @prashantgnet. Let me ask a few questions so I can help you debug.

  1. Did you run cf enable-ssh app_name before running cf ssh?
  2. How did you deploy your Cloud Foundry? Are you actually using cf-deployment (this repo) on vSphere, or did you use cf-release/diego-release? Which version of each repo are you using?
  3. Can you provide the output of bosh instances --ps? Please remember to display the output with three backticks, like this: ``` paste output here ```
prashantgnet commented 7 years ago

Please check my answers below....

1.  ssh enabled for app and space level as well.
[centos@bastion ~]$ cf enable-ssh cf-spring
ssh support is already enabled for 'cf-spring'

[centos@bastion ~]$ cf space-ssh-allowed test
ssh support is enabled in space 'test'
  1. Yes, I have deployed Cloud Foundry using cf-deployment on vSphere. I have deployed cf-deployment on Openstack as well and I am able to run cf ssh successfully on Openstack environment. I am facing cf ssh issue on vSphere environment. Output of bosh releases command
    Name                   Version   Commit Hash  
    binary-buildpack       1.0.13*   0bde2c77+    
    capi                   1.31.0*   63584ab8     
    cf-networking          1.1.0*    f081e3c      
    cf-smoke-tests         34*       47494b9      
    cflinuxfs2             1.135.0*  d97f0b46+    
    consul                 170*      06fd84c      
    diego                  1.20.0*   6f52b973     
    dotnet-core-buildpack  1.0.20*   dba15e78+    
    etcd                   112*      c97b2c3      
    garden-runc            1.9.0*    3f4312b5     
    go-buildpack           1.8.5*    3b12166a+    
    java-buildpack         3.18*     0b1dfe63+    
    loggregator            90.1*     0144d39      
    nats                   19*       2b004f21+    
    nodejs-buildpack       1.5.36*   ecd44abe+    
    php-buildpack          4.3.35*   01f6031f+    
    postgres               17*       b21260ee+    
    python-buildpack       1.5.19*   6e5bc32a+    
    routing                0.158.0*  cb9f094a     
    ruby-buildpack         1.6.41*   cc069b93+    
    staticfile-buildpack   1.4.9*    8c40e826+    
    statsd-injector        1.0.28*   4c0ffe4      
    uaa                    41*       83b20714  
  2. output of "bosh instances --ps" command
    
    Deployment 'cf'

Instance Process Process State AZ IPs
api/41f88ca9-f2d8-47b0-8fbd-a7361ad35ea0 - running z1 10.122.51.199
~ cloud_controller_ng running - -
~ cloud_controller_worker_local_1 running - -
~ cloud_controller_worker_local_2 running - -
~ consul_agent running - -
~ metron_agent running - -
~ nginx_cc running - -
~ policy-server running - -
~ route_registrar running - -
~ routing-api running - -
~ statsd_injector running - -
blobstore/0084f30b-0412-4383-960f-91b740578e26 - running z1 10.122.51.198
~ blobstore_nginx running - -
~ blobstore_url_signer running - -
~ consul_agent running - -
~ metron_agent running - -
~ route_registrar running - -
cc-bridge/aaa465c6-b74c-464a-94e6-d9dacc319c00 - running z1 10.122.51.204
~ cc_uploader running - -
~ consul_agent running - -
~ metron_agent running - -
~ nsync_bulker running - -
~ nsync_listener running - -
~ stager running - -
~ tps_listener running - -
~ tps_watcher running - -
cc-clock/d8238f15-1a48-4aab-bf02-eb045f60accb - running z1 10.122.51.203
~ cloud_controller_clock running - -
~ consul_agent running - -
~ metron_agent running - -
~ statsd_injector running - -
cc-worker/788e59a8-ca75-4868-9304-0964eaa8c65c - running z1 10.122.51.200
~ cloud_controller_worker_1 running - -
~ consul_agent running - -
~ metron_agent running - -
consul/11b3ba47-9644-40d5-88f9-2baaaf012b25 - running z1 10.122.51.191
~ consul_agent running - -
~ metron_agent running - -
diego-api/19fbd4c9-92f6-4f98-838b-cc5a220e6714 - running z1 10.122.51.196
~ bbs running - -
~ consul_agent running - -
~ metron_agent running - -
~ silk-controller running - -
diego-brain/db7064df-5060-4921-81f7-695c97f1c8ce - running z1 10.122.51.201
~ auctioneer running - -
~ consul_agent running - -
~ file_server running - -
~ metron_agent running - -
diego-cell/35bc15a0-db0a-4896-8a51-d5dbbc935107 - running z1 10.122.51.202
~ consul_agent running - -
~ garden running - -
~ metron_agent running - -
~ netmon running - -
~ rep running - -
~ route_emitter running - -
~ silk-daemon running - -
~ vxlan-policy-agent running - -
doppler/a714ab70-c10a-42dd-85ba-f1259719dd41 - running z1 10.122.51.194
~ consul_agent running - -
~ doppler running - -
~ metron_agent running - -
~ syslog_drain_binder running - -
etcd/773d4f89-90e5-4d47-87fa-0143f06cb2ac - running z1 10.122.51.193
~ consul_agent running - -
~ etcd running - -
~ etcd_consistency_checker running - -
~ etcd_metrics_server running - -
~ metron_agent running - -
log-api/7764a8d9-2d9c-48ae-b5ad-889382e0bf5f - running z1 10.122.51.205
~ consul_agent running - -
~ loggregator_trafficcontroller running - -
~ metron_agent running - -
~ route_registrar running - -
nats/5b29c30d-2ecf-4c3b-a8bd-dd38bb16e490 - running z1 10.122.51.192
~ consul_agent running - -
~ metron_agent running - -
~ nats running - -
postgres/ad09bb13-a8aa-4573-864c-bfb612f5d7ec - running z1 10.122.51.195
~ consul_agent running - -
~ metron_agent running - -
~ postgres running - -
router/66698df9-00af-4473-b7a7-295a36ea66e2 - running z1 10.122.51.143
~ consul_agent running - -
~ gorouter running - -
~ metron_agent running - -
~ ssh_proxy running - -
smoke-tests/b51d9988-9846-4295-8687-548b2d5fdc43 - - z1 -
uaa/3a1016f7-78a4-4227-9a60-53a48dec872b - running z1 10.122.51.197
~ consul_agent running - -
~ metron_agent running - -
~ route_registrar running - -
~ statsd_injector running - -
~ uaa running - -

93 instances

Succeeded



Thanks,
Prashant
dsabeti commented 7 years ago

Ok, very cool. We actually haven't tried deploying cf-deployment to vsphere yet. I've got a few more questions then:

  1. It looks like you moved the ssh_proxy from the diego-brain to the router job. Did you right an ops-file for that? Could you post the contents of it?
  2. What are you using for load balancing? GCP and AWS provide their own load balancers (i.e. ELBs), but vSphere typically requires that you bring your own. How is your load balancer configured?
  3. How is DNS configured for this environment? Specifically, what does ssh.YOUR_DOMAIN resolve to? What does that IP correspond to (load balancer, the router vm, etc.)?
prashantgnet commented 7 years ago

Please check my answers below....

  1. I have used "cf-deployment/operations/bosh-lite.yml" with very little modification for vSphere environment. This particular ops file contains the block which move "diego-brain" to the "router". I am not sure whether to keep it as it is or remove that block?

  2. I have not configured any load balancer for vSphere environment. I am redirecting all the traffic to router instance of cf-deployment.

  3. I have created DNS entry for "runtime.demo.com" and created wildcard entry for "*.runtime.demo.com". I have set the address of router interface to "runtime.demo.com", so all the traffic including "ssh.runtime.demo.com" goes towards router instance of cf-deployment.

I am getting following messages under router instance after running "cf ssh app_name" command.

router/66698df9-00af-4473-b7a7-295a36ea66e2:/var/vcap/sys/log/ssh_proxy# tail -f ssh_proxy.stdout.log 
{"timestamp":"1500284714.579789639","source":"ssh-proxy","message":"ssh-proxy.registration-runner.starting","log_level":1,"data":{"registration":{"Name":"ssh-proxy","Port":2222,"Check":{"TTL":"20s"},"Checks":null},"service":"ssh-proxy","session":"3"}}
{"timestamp":"1500284714.579970837","source":"ssh-proxy","message":"ssh-proxy.registration-runner.attempting-registering-service","log_level":1,"data":{"service":"ssh-proxy","session":"3"}}
{"timestamp":"1500284714.583348036","source":"ssh-proxy","message":"ssh-proxy.registration-runner.succeeded-registering-service","log_level":1,"data":{"service":"ssh-proxy","session":"3"}}
{"timestamp":"1500284714.584615946","source":"ssh-proxy","message":"ssh-proxy.registration-runner.poll-until-signaled.started","log_level":1,"data":{"service":"ssh-proxy","session":"3.1","update-interval":"10s"}}
{"timestamp":"1500284714.584750414","source":"ssh-proxy","message":"ssh-proxy.started","log_level":1,"data":{}}
{"timestamp":"1500284762.419135094","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","log_level":2,"data":{"error":"no auth passed yet","user":"cf:12d8da0a-42a2-4090-a184-a3d3087a3db0/0"}}
{"timestamp":"1500284762.419444799","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-starting","log_level":1,"data":{"session":"5"}}
{"timestamp":"1500284767.419733286","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.exchange-access-code-for-token.request-failed","log_level":2,"data":{"error":"Post https://uaa.demo.com/oauth/token: dial tcp: i/o timeout","session":"5.1"}}
{"timestamp":"1500284767.419839382","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-finished","log_level":1,"data":{"session":"5"}}
{"timestamp":"1500284767.419859171","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","log_level":2,"data":{"error":"Authentication failed","user":"cf:12d8da0a-42a2-4090-a184-a3d3087a3db0/0"}}
^C
router/66698df9-00af-4473-b7a7-295a36ea66e2:/var/vcap/sys/log/ssh_proxy#

Kindly let me know in case any changes are required for vSphere environment or any other details required for the same.

Note: I did the same thing on Openstack environment as well and "cf ssh app_name" is working fine. I have used bosh-lite.yml file with minor modifications on Openstack environment. i have kept the block for moving of diego-brain to router as it is.

Thanks, Prashant

prashantgnet commented 7 years ago

Issue is sorted out! I have wrong entry in my cloud-config file for DNS server.
I still have query related to diego-brain, do I need to move it router or not?

Thanks a lot David for your support.

Thanks, Prashant

dsabeti commented 7 years ago

@prashantgnet It's probably fine to keep it there.