node session id does not match event log

[programmerq]

Current behavior:

When connected to an SSH session, it is possible to grab your session id by running teleport status:

$ teleport status
User ID     : jeff, logged in as ubuntu from 10.3.3.2 443 35524
Cluster Name: mycluster
Host UUID   : 23bc35d7-10f5-421f-9ef4-890af45e3166
Session ID  : a1f2b7f5-c6b3-4a79-8889-d125ff1684f7
Session URL : https://<proxyhost>:3080/web/cluster/other/console/session/a1f2b7f5-c6b3-4a79-8889-d125ff1684f7

If using proxy recording mode (and I suspect moderated sessions too), this session id does not show up in the event log, and does not work for session playback. The session id that appears in the proxy logs is the one that will appear in the audit events and work with tsh play:

{"ei":0,"event":"session.start","uid":"df21c9ae-5506-4cd2-8e45-68ef4f1001c9","code":"T2000I","time":"2023-01-10T22:23:57.642Z","cluster_name":"mycluster","user":"jeff","login":"ubuntu","sid":"d46d710b-24b9-4f86-af68-d7755636b94b","namespace":"default","server_id":"23bc35d7-10f5-421f-9ef4-890af45e3166.other","server_hostname":"mynode","server_addr":"@local-node","forwarded_by":"31486a8f-ed19-45e7-9610-ddb916493e52","addr.remote":"10.5.5.3:65066","proto":"ssh","size":"148:37","initial_command":[""],"session_recording":"proxy"}

Additionally, there does not appear to be any reference on the node nor on the proxy to establish a relationship between the session id as seen on the proxy, and the session id as seen on the node.

Expected behavior

The teleport status output, and teleport node agent logs should have a session id that will appear in the audit events and work for session playback.

At the very least, it should be possible to relate the user-visible session id (as seen in teleport status or the teleport node logs) with the session id in the event log. Ideally, one would be able to search either log for either session id and get the relevant matches.

Bug details:

• Teleport version Teleport Enterprise v11.2.0 git:api/v11.2.0-0-g7591b8d go1.19.4
• Recreation steps - see above
• Debug logs - see above

[xacrimon]

note for the implementor/assignee: due to the way our SSH session proxying code works to allow for proxy-recording, the codepath is the same regardless if you are a proxy or not and you have to check the component to figure out if you are a node or proxy (see lib/srv/sess.go). As a result of this, the proxy also generates a session ID as the code doesn't know it's running in proxy mode, so there's two of these.