search

adiom-data / dsync

Database synchronization tool
https://adiom.io
GNU Affero General Public License v3.0
4 stars 1 forks source link

Bug in CosmosDeletes interruption #25

Open alex-thc opened 1 month ago

alex-thc commented 1 month ago 
^CJul 29 22:15:40.175 ERR Failed to delete document from collection: connection(destinationdemo-shard-00-01.jnvpr.mongodb.net:27017[-29]) incomplete read of message header: context canceled
Jul 29 22:15:40.175 ERR Failed to process data message: connection(destinationdemo-shard-00-01.jnvpr.mongodb.net:27017[-29]) incomplete read of message header: context canceled
Jul 29 22:15:40.175 ERR Failed to delete document from collection: canceled while checking out a connection from connection pool: context canceled; total connections: 3, maxPoolSize: 100, idle connections: 0, wait duration: 416ns
Jul 29 22:15:40.175 ERR Failed to process data message: canceled while checking out a connection from connection pool: context canceled; total connections: 3, maxPoolSize: 100, idle connections: 0, wait duration: 416ns
Jul 29 22:15:40.175 INF Flow with ID: 86fc02ddf96fdf63 is done
Jul 29 22:15:40.175 ERR Failed to delete document from collection: connection(destinationdemo-shard-00-01.jnvpr.mongodb.net:27017[-30]) incomplete read of message header: context canceled
Jul 29 22:15:40.176 ERR Failed to process data message: connection(destinationdemo-shard-00-01.jnvpr.mongodb.net:27017[-30]) incomplete read of message header: context canceled
panic: send on closed channel

goroutine 1086 [running]:
github.com/adiom-data/dsync/connector/connectorCosmos.(*CosmosConnector).generateDeleteMessages(0x105237420?, 0x140029b4300, 0x14000110a20)
    /Users/alexander/work/adiom-data/dsync/connector/connectorCosmos/checkdeletes.go:257 +0x100
github.com/adiom-data/dsync/connector/connectorCosmos.(*CosmosConnector).checkForDeletes_sync(0x14000338420, {0x140007d1f10?, 0x0?}, {{0x1400031a450?, 0x0?, 0x14000360001?}}, 0x14000110a20)
    /Users/alexander/work/adiom-data/dsync/connector/connectorCosmos/checkdeletes.go:77 +0x2a0
github.com/adiom-data/dsync/connector/connectorCosmos.(*CosmosConnector).StartReadToChannel.func1.1()
    /Users/alexander/work/adiom-data/dsync/connector/connectorCosmos/connectorcosmos.go:250 +0x144
created by github.com/adiom-data/dsync/connector/connectorCosmos.(*CosmosConnector).StartReadToChannel.func1 in goroutine 362
    /Users/alexander/work/adiom-data/dsync/connector/connectorCosmos/connectorcosmos.go:237 +0x248
alex-thc commented 1 month ago

This is because we eagerly shutdown on Ctrl+C and don't let things in flight to finish. Proposed changes are in the linked branch