`TestDynamicClientReuse` flakiness

[rosstimothy]

Failure

Link(s) to logs

• https://github.com/gravitational/teleport/actions/runs/11106415967/job/30854698081?pr=46875

Relevant snippet

=== FAIL: lib/service TestDynamicClientReuse (2.02s)
    testing.go:1232: TempDir RemoveAll cleanup: unlinkat /tmp/TestDynamicClientReuse2126280772/001: directory not empty

[codingllama]

Another at https://github.com/gravitational/teleport/actions/runs/11132134122/job/30935280214 (and I think a couple other times today too).

[GavinFrazar]

Hit on master (checked that the branch contains the fix): https://github.com/gravitational/teleport/actions/runs/11169236475/job/31049479095?pr=47152

=== FAIL: lib/service TestDynamicClientReuse (2.19s)
    testing.go:1232: TempDir RemoveAll cleanup: unlinkat /tmp/TestDynamicClientReuse3180591090/001: directory not empty
{"caller":"service/supervisor.go:411","component":"proc:4","event":"ServiceExitedWithError","level":"debug","message":"Broadcasting event.","pid":"123750.4","timestamp":"2024-10-03T20:35:46Z"}
{"caller":"service/supervisor.go:438","component":"proc:4","level":"debug","message":"Teleport not yet ready: still waiting for ProxySSHReady","pid":"123750.4","timestamp":"2024-10-03T20:35:46Z"}
{"caller":"service/supervisor.go:255","component":"proc:4","level":"debug","message":"Service is completed and removed.","pid":"123750.4","service":"proxy.init","timestamp":"2024-10-03T20:35:46Z"}
{"timestamp":"2024-10-03T20:35:46Z","level":"debug","caller":"service/connect.go:1131","message":"Connected to Auth Server with direct connection.","component":"proc:14","pid":"123750.14","component":"proc:15","pid":"123750.15","auth_addrs":["127.0.0.1:39711"]}
{"timestamp":"2024-10-03T20:35:46Z","level":"warning","caller":"service/connect.go:181","message":"This instance is too new. Using a newer major version than the Auth server is unsupported and may impair functionality.","component":"proc:14","pid":"123750.14","component":"proc:15","pid":"123750.15","version":18,"auth_version":17,"supported_versions":[17,16]}
{"timestamp":"2024-10-03T20:35:46Z","level":"info","caller":"service/connect.go:1093","message":"features loaded from auth server","component":"proc:14","pid":"123750.14","component":"proc:15","pid":"123750.15","identity":"Instance","features":"Kubernetes:true App:true DB:true Desktop:true DeviceTrust:<> AccessRequests:<> AccessList:<> AccessMonitoring:<> Policy:<> SupportType:SUPPORT_TYPE_FREE JoinActiveSessions:true entitlements:<key:\"AccessLists\" value:<> > entitlements:<key:\"AccessMonitoring\" value:<> > entitlements:<key:\"AccessRequests\" value:<> > entitlements:<key:\"App\" value:<enabled:true > > entitlements:<key:\"CloudAuditLogRetention\" value:<> > entitlements:<key:\"DB\" value:<enabled:true > > entitlements:<key:\"Desktop\" value:<enabled:true > > entitlements:<key:\"DeviceTrust\" value:<> > entitlements:<key:\"ExternalAuditStorage\" value:<> > entitlements:<key:\"FeatureHiding\" value:<> > entitlements:<key:\"HSM\" value:<> > entitlements:<key:\"Identity\" value:<> > entitlements:<key:\"JoinActiveSessions\" value:<enabled:true > > entitlements:<key:\"K8s\" value:<enabled:true > > entitlements:<key:\"LicenseAutoUpdate\" value:<> > entitlements:<key:\"MobileDeviceManagement\" value:<> > entitlements:<key:\"OIDC\" value:<> > entitlements:<key:\"OktaSCIM\" value:<> > entitlements:<key:\"OktaUserSync\" value:<> > entitlements:<key:\"Policy\" value:<> > entitlements:<key:\"SAML\" value:<> > entitlements:<key:\"SessionLocks\" value:<> > entitlements:<key:\"UpsellAlert\" value:<> > entitlements:<key:\"UsageReporting\" value:<> > "}
{"timestamp":"2024-10-03T20:35:46Z","level":"info","caller":"service/connect.go:506","message":"The process successfully wrote the credentials and state to the disk.","component":"proc:14","pid":"123750.14","component":"proc:15","pid":"123750.15","identity":"Instance"}
{"timestamp":"2024-10-03T20:35:46Z","level":"debug","caller":"service/connect.go:197","message":"Client successfully connected to cluster","component":"proc:14","pid":"123750.14","component":"proc:15","pid":"123750.15","client_identity":"Identity(Instance, cert(30aeb5ac-6070-4f6f-a0f0-cbbaa6f152b1.14061f51b603 issued by 14061f51b603:290670487730283783347721006104563000151),trust root(14061f51b603:290670487730283783347721006104563000151))"}

[rosstimothy]

https://github.com/gravitational/teleport/actions/runs/11182269625/job/31088180434?pr=46899

=== Failed
=== FAIL: lib/service TestDynamicClientReuse (2.39s)
    testing.go:1232: TempDir RemoveAll cleanup: unlinkat /tmp/TestDynamicClientReuse1060631134/001: directory not empty