search

cloudfoundry / stratos

Stratos: Web-based Management UI for Cloud Foundry and Kubernetes
Apache License 2.0
251 stars 132 forks source link

No SSH connection possible "Error connecting to web socket" #4870

Open timonbirk-witt opened 3 years ago

timonbirk-witt commented 3 years ago

Stratos Version

4.4.0

Frontend Deployment type

Backend (Jet Stream) Deployment type

Expected behaviour

SSH access to apps is possible. Stratos uses the correct id.

Actual behaviour

For several apps we can't create a ssh connection (via instances -> SSH) with stratos. We only get the error message "Error connecting to web socket".

After some debugging we discovered that Stratos is requesting with a "wrong" application id. Here are the logentries from the scheduler/ssh-proxy service in cloud foundry:

{​​​​​​​"timestamp":"2021-02-09T10:06:34.516532713Z","level":"error","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","data":{​​​​​​​"error":"ssh: no auth passed yet","user":"cf:94c2a04d-eabc-44f9-a31f-0e9d666e9aec/0@ssh.mycompany"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:06:34.517828483Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-starting","data":{​​​​​​​"session":"264"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:06:34.858569794Z","level":"error","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.fetching-app-failed","data":{​​​​​​​"ResponseBody":{​​​​​​​}​​​​​​​,"StatusCode":"404 Not Found","app":"94c2a04d-eabc-44f9-a31f-0e9d666e9aec/0","error":"Fetching application data failed","principal":"99d98a36-8c29-416d-a424-f19289409848","session":"264","username":"me@mycompany"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:06:34.858785507Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-finished","data":{​​​​​​​"session":"264"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:06:34.858832877Z","level":"error","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","data":{​​​​​​​"error":"Fetching application data failed","user":"cf:94c2a04d-eabc-44f9-a31f-0e9d666e9aec/0@ssh.mycompany"}​​​​​​​}​​​​​​​

So the app id "94c2a04d-eabc-44f9-a31f-0e9d666e9aec/0" could not be found.

Running the "cf cli" command for the same application results in

{"timestamp":"2021-02-09T10:26:08.465319920Z","level":"error","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","data":{​​​​​​​"error":"ssh: no auth passed yet","user":"cf:4e874674-df2b-4eb1-ab62-ca90a3f4db91/0"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.467275889Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-starting","data":{​​​​​​​"session":"738"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.751682714Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.app-access-success","data":{​​​​​​​"app":"4e874674-df2b-4eb1-ab62-ca90a3f4db91/0","principal":"99d98a36-8c29-416d-a424-f19289409848","session":"738","username":"me@mycompany"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.751719507Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-finished","data":{​​​​​​​"session":"738"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.751752850Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.authentication-attempted","data":{​​​​​​​"user":"cf:4e874674-df2b-4eb1-ab62-ca90a3f4db91/0"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.767173858Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.handle-connection.new-client-conn.connected-to-backend","data":{​​​​​​​"backend-address":"10.176.213.8:61075","session":"737.1"}​​​​​​​}​​​​​​​
{​​​​​​​"timestamp":"2021-02-09T10:26:08.773429884Z","level":"info","source":"ssh-proxy","message":"ssh-proxy.handle-connection.from-client.proxy-global-requests.started","data":{​​​​​​​"session":"737.2.1"}​​​​​​​}​​​​​​​

and I can ssh into the app.

Seems like the cf cli (version 7.1.0) is using another id.

Having a closer look into the application we can see that cf cli is using the "process id" (4e874674-df2b-4eb1-ab62-ca90a3f4db91) and stratos the app id (94c2a04d-eabc-44f9-a31f-0e9d666e9aec).

VCAP_APPLICATION={"cf_api":"https://api.mycompany","limits":{"fds":16384,"mem":384,"disk":1024},"application_name":"myapp","application_uris":["myapp.mycompany"],"name":"myapp","space_name":"myspace","space_id":"d64180a4-e607-419f-92f9-d95c96ea7759","organization_id":"8611e8b0-ff63-4b76-9017-df69e8166331","organization_name":"myorg","uris":["myapp.mycompany"],"process_id":"4e874674-df2b-4eb1-ab62-ca90a3f4db91","process_type":"web","application_id":"94c2a04d-eabc-44f9-a31f-0e9d666e9aec","version":"43abc7e3-f40a-43b6-b8c4-9a88f0cd4ebd","application_version":"43abc7e3-f40a-43b6-b8c4-9a88f0cd4ebd"}

Maybe the difference is because the app was deployed with the CF 7 rolling app deployment (cf push APP-NAME --strategy rolling)?

Other applications (which we can access with stratos and cf cli) don't have different ids as app id and process id.

richard-cox commented 3 years ago

CF Apps deployed via CF CLI v7 will use v3 of the CF API (which brings in/exposes the process id). We've been waiting a long time for v3 to be completed and it's not currently supported by Stratos.