search

gravitational / teleport

The easiest, and most secure way to access and protect all of your infrastructure.
https://goteleport.com
GNU Affero General Public License v3.0
17.42k stars 1.74k forks source link

Connection via ssh using Paramiko (Python) is generating an error in DynamoDB on AWS #42652

Closed lucasfellipe3g closed 2 months ago

lucasfellipe3g commented 4 months ago

Expected behavior: When I connect via ssh, it doesn't generate a conflict error in DynamoDB...

Current behavior: When I connect to a virtual environment using a Python code and Paramiko Library, I received the following log and error:

Jun 07 19:22:40 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: 2024-06-07T19:22:40Z INFO [AUDIT]     session.leave cluster_name:teleport code:T2003I ei:2 event:session.leave login:zerum namespace:default private_key_policy:none server_hostname:lynx-dev server_id:b25d88f8-e0d0-444b-bf9f-2bae5f18197d ip:192.168.100.118 label:Development spec:lab/lynx:dev type:Metal sid:0603f806-6680-4212-8cc7-bbab2d123a22 time:2024-06-07T19:22:40.373Z uid:47ab919c-6229-450d-9062-ef6883fad896 user:lucas.fellipe user_kind:1 events/emitter.go:288
Jun 07 19:22:40 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: 2024-06-07T19:22:40Z INFO [AUDIT]     session.data addr.remote:186.193.13.167:25192 cluster_name:teleport code:T2006I ei:2.147483646e+09 event:session.data login:zerum namespace:default private_key_policy:none rx:3010 server_hostname:lynx-dev server_id:b25d88f8-e0d0-444b-bf9f-2bae5f18197d sid:0603f806-6680-4212-8cc7-bbab2d123a22 time:2024-06-07T19:22:40.4Z tx:3592 uid:49817335-3e3d-4671-83c3-f9450d5bf413 user:lucas.fellipe user_kind:1 events/emitter.go:288
Jun 07 19:22:43 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: 2024-06-07T19:22:43Z INFO [APP:WEB]   Round trip: GET /internal/security/users, code: 200, duration: 20.370629ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server: reverseproxy/reverse_proxy.go:223
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: 2024-06-07T19:22:45Z INFO [AUDIT]     exec addr.local:192.168.100.118:39220 addr.remote:186.193.13.167:25192 cluster_name:teleport code:T3002I command:echo "Hello World!" ei:0 event:exec exitCode:0 login:zerum namespace:default private_key_policy:none server_hostname:lynx-dev server_id:b25d88f8-e0d0-444b-bf9f-2bae5f18197d sid:0603f806-6680-4212-8cc7-bbab2d123a22 time:2024-06-07T19:22:45.342Z uid:f882f6cb-7ddd-47fc-b68c-bf8dbc742cce user:lucas.fellipe user_kind:1 events/emitter.go:288
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: 2024-06-07T19:22:45Z ERRO [DYNAMODB]  Conflict on event session_id and event_index error:[
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: ERROR REPORT:
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: Original Error: *trace.AlreadyExistsError ConditionalCheckFailedException: The conditional request failed
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: {
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   RespMetadata: {
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:     StatusCode: 400,
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:     RequestID: "HI78O9L6NULA9K68O62BNF3ST7VV4KQNSO5AEMVJF66Q9ASUAAJG"
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   },
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   Message_: "The conditional request failed"
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: }
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: Stack Trace:
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:1061 github.com/gravitational/teleport/lib/events/dynamoevents.convertError
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:455 github.com/gravitational/teleport/lib/events/dynamoevents.(*Log).putAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/events/dynamoevents/dynamoevents.go:376 github.com/gravitational/teleport/lib/events/dynamoevents.(*Log).EmitAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/events/emitter.go:310 github.com/gravitational/teleport/lib/events.(*MultiEmitter).EmitAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/events/emitter.go:178 github.com/gravitational/teleport/lib/events.(*CheckingEmitter).EmitAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/auth/auth_with_roles.go:3861 github.com/gravitational/teleport/lib/auth.(*ServerWithRoles).EmitAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/auth/grpcserver.go:195 github.com/gravitational/teleport/lib/auth.(*GRPCServer).EmitAuditEvent
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/api@v0.0.0/client/proto/authservice.pb.go:22714 github.com/gravitational/teleport/api/client/proto._AuthService_EmitAuditEvent_Handler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/auth/middleware.go:478 github.com/gravitational/teleport/lib/auth.(*Middleware).withAuthenticatedUserUnaryInterceptor
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1186 google.golang.org/grpc.getChainUnaryHandler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/lib/limiter/limiter.go:155 github.com/gravitational/teleport/lib/auth.(*Middleware).UnaryInterceptors.(*Limiter).UnaryServerInterceptorWithCustomRate.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1186 google.golang.org/grpc.getChainUnaryHandler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/api@v0.0.0/metadata/metadata.go:75 github.com/gravitational/teleport/api/metadata.UnaryServerInterceptor
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1186 google.golang.org/grpc.getChainUnaryHandler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/api@v0.0.0/utils/grpc/interceptors/errors.go:76 github.com/gravitational/teleport/api/utils/grpc/interceptors.GRPCServerUnaryErrorInterceptor
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1186 google.golang.org/grpc.getChainUnaryHandler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.1/interceptors/server.go:22 github.com/gravitational/teleport/lib/auth.(*Middleware).UnaryInterceptors.(*ServerMetrics).UnaryServerInterceptor.UnaryServerInterceptor.func2
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1186 google.golang.org/grpc.getChainUnaryHandler.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.49.0/interceptor.go:326 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1177 google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         github.com/gravitational/teleport/api@v0.0.0/client/proto/authservice.pb.go:22716 github.com/gravitational/teleport/api/client/proto._AuthService_EmitAuditEvent_Handler
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1369 google.golang.org/grpc.(*Server).processUnaryRPC
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1780 google.golang.org/grpc.(*Server).handleStream
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         google.golang.org/grpc@v1.63.2/server.go:1019 google.golang.org/grpc.(*Server).serveStreams.func2.1
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:         runtime/asm_amd64.s:1650 runtime.goexit
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: User Message: ConditionalCheckFailedException: The conditional request failed
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: {
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   RespMetadata: {
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:     StatusCode: 400,
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:     RequestID: "HI78O9L6NULA9K68O62BNF3ST7VV4KQNSO5AEMVJF66Q9ASUAAJG"
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   },
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]:   Message_: "The conditional request failed"
Jun 07 19:22:45 ip-172-31-3-248.sa-east-1.compute.internal teleport[14771]: }] event_index:0 event_type:exec session_id:0603f806-6680-4212-8cc7-bbab2d123a22 dynamoevents/dynamoevents.go:470

My python code:

import os

import paramiko

hostname = 'pluto.teleport'
username = 'pluto'

tsh_user = 'flask'

ssh_config = paramiko.SSHConfig()
ssh_config.parse(open(os.path.expanduser('~/.ssh/config')))
host = ssh_config.lookup(hostname)

client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

private_key = paramiko.RSAKey.from_private_key_file(f'/home/piplup/.tsh/keys/teleport.amz.company.com/{tsh_user}')
private_key.load_certificate(f'/home/piplup/.tsh/keys/teleport.amz.company.com/{tsh_user}-ssh/teleport-cert.pub')

client.connect(host['hostname'], username=username, sock=paramiko.ProxyCommand(host.get('proxycommand')), pkey=private_key, port=int(host.get('port')))

if client.get_transport().is_active():
    print('Connection is active')
else:
    print('Connection is not active')

stdin, stdout, stderr = client.exec_command('echo "Hello World!"')
print(stdout.read().decode('utf-8'))

client.close()

Bug details:

zmb3 commented 2 months ago

This was fixed by #40854.