Closed chakaz closed 1 week ago
it expects that this->OnBreakCb(mask)
will call on facade::Connection*
, aka dragonfly connection. instead it's being called on util::Connection, i.e. AFTER facade::~Connection
destructor has finished, most likely during the destruction of util::Connection
.
after inspecting it further, dragonfly_connection.cc:768] Unexpected event 8192
means that the event from iouring socket arrived after
connection context was destroyed, proving without doubt that event notifications arrive after they have been cancelled (i.e. after CancelPoll has been called and finished running).
This means, we must redesign the socket polling interface in such way that the socket could ignore events coming for callback that has been unregistered.
in other words we should use some sort of refcounting mechanism
8192 is EPOLLHUP
event
@chakaz do you remember if you run locally both the client and the server? what was the client?
Definitely locally
The client was python async library
I created many connections (I think that after each connection I issued a PING
also)
do you still have this code? it could be interesting to adapt it into a regtest
Yes, it's here: https://github.com/dragonflydb/dragonfly/pull/2219/files (was never merged)
IIRC you should run it with --action=connections --keys=30000
(or maybe 25k?)
I ran Dragonfly (debug build) and created many connections to it (almost 30k). At some point I got an error from UB sanitizer (see below).
Notes:
export UBSAN_OPTIONS print_stacktrace=1
to print the call stack, it's a nice trick to knowUnexpected event
errors are related. I included them as they probably are, but maybe notFWIW, line 405 in
dragonfly_connection.cc
isSo to my understanding somehow this callback is called when the address that was pointed to be
this
was changed or while it is being destructed (because oddly the object is autil::Connection
while not being aConnection
:thinking:)