Closed mafintosh closed 3 years ago
stack trace from a case using 100 nodes over mininet, with 25% of nodes being destroyed during a scale test (this error occurs roughly 2-3 times in such a scenario), where destroy
is guarded but send
is not:
Error [ERR_SOCKET_DGRAM_NOT_RUNNING]: Not running
at Socket._healthCheck (dgram.js:638:11)
at Socket.send (dgram.js:435:8)
at IO.send (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:205:17)
at IO._requestImmediately (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:166:10)
at IO.queryImmediately (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:262:10)
at ping (/home/ubuntu/grenache-grape/node_modules/dht-rpc/index.js:259:16)
at HyperDHT._reping (/home/ubuntu/grenache-grape/node_modules/dht-rpc/index.js:254:5)
at HyperDHT._onnodeping (/home/ubuntu/grenache-grape/node_modules/dht-rpc/index.js:241:29)
at emitTwo (events.js:126:13)
at KBucket.emit (events.js:214:7)
at KBucket.add (/home/ubuntu/grenache-grape/node_modules/k-bucket/index.js:186:12)
at Object.afterPing [as callback] (/home/ubuntu/grenache-grape/node_modules/dht-rpc/index.js:265:19)
at IO._finish (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:132:9)
at IO._cancel (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:173:10)
at IO.destroy (/home/ubuntu/grenache-grape/node_modules/dht-rpc/lib/io.js:218:43)
at HyperDHT.destroy (/home/ubuntu/grenache-grape/node_modules/dht-rpc/index.js:176:14)
at EventEmitter.tapenet.once ([h68-test]:84:18)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
at EventEmitter.emit (events.js:208:7)
at EventEmitter.<anonymous> ([eval]:42:39)
at emitTwo (events.js:126:13)
at EventEmitter.emit (events.js:214:7)
at DestroyableTransform.<anonymous> (/home/ubuntu/grenache-grape/test/mininet/node_modules/mininet/host.js:25:11)
at emitOne (events.js:116:13)
at DestroyableTransform.emit (events.js:211:7)
at addChunk (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_readable.js:291:12)
at readableAddChunk (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_readable.js:278:11)
at DestroyableTransform.Readable.push (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_readable.js:245:10)
at DestroyableTransform.Transform.push (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_transform.js:148:32)
at push (/home/ubuntu/grenache-grape/test/mininet/node_modules/split2/index.js:52:10)
at DestroyableTransform.transform [as _transform] (/home/ubuntu/grenache-grape/test/mininet/node_modules/split2/index.js:33:5)
at DestroyableTransform.Transform._read (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_transform.js:184:10)
at DestroyableTransform.Transform._write (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_transform.js:172:83)
at doWrite (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_writable.js:428:64)
at writeOrBuffer (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_writable.js:417:5)
at DestroyableTransform.Writable.write (/home/ubuntu/grenache-grape/test/mininet/node_modules/readable-stream/lib/_stream_writable.js:334:11)
at Socket.ondata (_stream_readable.js:639:20)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at Pipe.onread (net.js:601:20)
I think we fixed this
Some part of the code seems to trigger an send after destroy in io. We should trace that down and fix it.