EventStore / replicator

Real-time replication tool
https://replicator.eventstore.org
Apache License 2.0
20 stars 13 forks source link

Connection disposed exception with TCP event reader #60

Open shaan1337 opened 1 year ago

shaan1337 commented 1 year ago

Describe the bug A customer reported seeing the following stack trace:

{"@t":"2022-12-23T03:13:15.6525085Z","@m":"Starting from a previously known checkpoint \"Position { EventNumber = 61876, EventPosition = 1077218785 }\"","@i":"7734e6c4","LastKnown":"Position { EventNumber = 61876, EventPosition = 1077218785 }","SourceContext":"EventStore.Replicator.FileCheckpointStore"}
{"@t":"2022-12-23T03:13:15.6525645Z","@m":"Reading from \"Position { EventNumber = 61876, EventPosition = 1077218785 }\"","@i":"91138b7a","Position":"Position { EventNumber = 61876, EventPosition = 1077218785 }","SourceContext":"EventStore.Replicator.Read.ReaderPipe"}
{"@t":"2022-12-23T03:13:15.6527311Z","@m":"Reader error","@i":"450d20cc","@l":"Error","@x":"System.ObjectDisposedException: Cannot access a disposed object.\nObject name: 'ES-2e381aa6-ae42-4b1d-aa8f-7f87e283d16a'.\n at EventStore.Replicator.Esdb.Tcp.TcpEventReader.ReadEvents(Position fromPosition, Func`2 next, CancellationToken cancellationToken) in /app/src/EventStore.Replicator.Esdb.Tcp/TcpEventReader.cs:line 54\n at EventStore.Replicator.Read.ReaderPipe.<>c__DisplayClass1_0.<<-ctor>g__Reader|1>d.MoveNext() in /app/src/EventStore.Replicator/Read/ReaderPipe.cs:line 60","SourceContext":"EventStore.Replicator.Read.ReaderPipe"}
{"@t":"2022-12-23T03:13:15.6528929Z","@m":"Reader stopped","@i":"49a9146f","SourceContext":"EventStore.Replicator.Read.ReaderPipe"}
{"@t":"2022-12-23T03:13:15.6529162Z","@m":"Storing the last known checkpoint","@i":"228021e9","SourceContext":"EventStore.Replicator.Replicator"}
{"@t":"2022-12-23T03:13:15.6531668Z","@m":"Will restart in 5 sec","@i":"d2d2cf9b","SourceContext":"EventStore.Replicator.Replicator"}

The crash seems to be coming from this line: https://github.com/EventStore/replicator/blob/2f2957221833b1a15041383de588532ee156cc91/src/EventStore.Replicator.Esdb.Tcp/TcpEventReader.cs#L54

It appears that _connection object's state was ConnectionState.Closed when calling ConnectAsync() (not sure why, maybe it was in the middle of a reconnection but this hypothesis doesn't seem to make sense since based on a quick glance at the code it appears that it's the first time we're calling ConnectAsync to establish the connection): https://github.com/EventStore/EventStoreDB-Client-Dotnet-Legacy/blob/f29fb7f0c89bd1a7b7390379373b7266261da934/src/EventStore.ClientAPI/Internal/EventStoreConnectionLogicHandler.cs#L112

DEV-74