Problem disconnecting a GetRange request

daniel-zullo commented 5 years ago

The scenario is as follows:

The app disconnects a DLS neo GetRange request due to a long request timeout
- Dls request failed due to a specific node
- Expected non-zero errno after failure (connection hung up on read)
Received request stopped notification on the app side
App retries the request after 5 seconds
Request finished successfully
App processed 4 more requests successfully
and suddenly the app crashes due to dlsproto.client.request.internal.GetRange.GetRangeHandler.forceStopRequest()

daniel-zullo commented 5 years ago

#0  0x00000000013cb870 in dlsproto.client.request.internal.GetRange.GetRangeHandler.forceStopRequest() (this=...)
    at ./submodules/swarm/src/swarm/neo/request/RequestEventDispatcher.d:481
#1  0x00000000014c89ea in dlsproto.client.internal.SharedResources.SharedResources.Timer.timerDg() (this=0x7ff5edb63680)
    at ./submodules/dlsproto/src/dlsproto/client/internal/SharedResources.d:412
#2  0x00000000013dbd5b in ocean.io.select.client.TimerEvent.TimerEvent.handle_(ulong) (this=0x7ff5ed9eaa00, n=140720752023552)
    at ./submodules/ocean/src/ocean/io/select/client/TimerEvent.d:99
#3  0x00000000013dbf50 in ocean.io.select.client.TimerEvent.ITimerEvent.handle(ocean.sys.Epoll.epoll_event_t.Event) (this=0x7ff5ed9eaa00, event=3986598400)
    at ./submodules/ocean/src/ocean/io/select/client/TimerEvent.d:360
#4  0x00000000014cfa3d in ocean.io.select.selector.SelectedKeysHandler.SelectedKeysHandler.handleSelectedKey(ocean.sys.Epoll.epoll_event_t, scope bool(Exception) delegate) (this=0x0, unhandled_exception_hook=..., key=...) at ./submodules/ocean/src/ocean/io/select/selector/SelectedKeysHandler.d:170
#5  0x00000000014cf9d1 in ocean.io.select.selector.SelectedKeysHandler.SelectedKeysHandler.opCall(ocean.sys.Epoll.epoll_event_t[], scope bool(Exception) delegate) (this=0x7ff6291ccb00, unhandled_exception_hook=..., selected_set=...) at ./submodules/ocean/src/ocean/io/select/selector/SelectedKeysHandler.d:136
#6  0x00000000014d0b03 in ocean.io.select.EpollSelectDispatcher.EpollSelectDispatcher.select(bool) (this=0x7ff6291cd680, exit_asap=false)
    at ./submodules/ocean/src/ocean/io/select/EpollSelectDispatcher.d:836
#7  0x00000000014d0936 in ocean.io.select.EpollSelectDispatcher.EpollSelectDispatcher.eventLoop(scope bool() delegate, scope bool(Exception) delegate) (
    this=0x7ff6291cd680, unhandled_exception_hook=..., select_cycle_hook=...) at ./submodules/ocean/src/ocean/io/select/EpollSelectDispatcher.d:749
#8  0x0000000001354ccc in ocean.task.Scheduler.Scheduler.eventLoop() (this=0x7ff6291cd580) at ./submodules/ocean/src/ocean/task/Scheduler.d:128

daniel-zullo commented 5 years ago

The bug triggered again using DMD64 D Compiler v2.078.3-2-g91096fe, dlsproto v14.1.2, ocean v4.3.4 and swarm v5.2.1

#0  0x0000000000c86c20 in dlsproto.client.request.internal.GetRange.GetRangeHandler.forceStopRequest() (this=...)
    at ./submodules/swarm/src/swarm/neo/request/RequestEventDispatcher.d:483
#1  0x0000000000d80bda in _D8dlsproto6client8internal15SharedResourcesQr5Timer7timerDgMFZb (this=0x7ffa46c3b940)
    at ./submodules/dlsproto/src/dlsproto/client/internal/SharedResources.d:413
#2  0x0000000000c972d7 in _D5ocean2io6select6client10TimerEventQm7handle_MFmZb (this=0x7ffa46ebda80, n=140731174349920)
    at ./submodules/ocean/src/ocean/io/select/client/TimerEvent.d:99
#3  0x0000000000c974a8 in _D5ocean2io6select6client10TimerEvent11ITimerEvent6handleMFEQCg3sys5Epoll13epoll_event_t5EventZb (
    this=0x7ffa46ebda80, event=1189862016) at ./submodules/ocean/src/ocean/io/select/client/TimerEvent.d:360
#4  0x0000000000d87355 in _D5ocean2io6select8selector19SelectedKeysHandlerQv17handleSelectedKeyMFSQCs3sys5Epoll13epoll_event_tMDFC9ExceptionZbZv (this=0xffffffffffffffff, unhandled_exception_hook=0x0, __unhandled_exception_hook_8=0x0, key=...)
    at ./submodules/ocean/src/ocean/io/select/selector/SelectedKeysHandler.d:188
#5  0x0000000000d872e9 in _D5ocean2io6select8selector19SelectedKeysHandlerQv6opCallMFASQCh3sys5Epoll13epoll_event_tMDFC9ExceptionZbZv (
    this=0x7ffa4bc7ed00, unhandled_exception_hook=..., selected_set=3, __selected_set_8=0)
    at ./submodules/ocean/src/ocean/io/select/selector/SelectedKeysHandler.d:136
#6  0x0000000000d883f3 in _D5ocean2io6select21EpollSelectDispatcherQxQBgMFbZk (this=0x7ffa4bc7f700, exit_asap=false)
    at ./submodules/ocean/src/ocean/io/select/EpollSelectDispatcher.d:836
#7  0x0000000000d881ed in _D5ocean2io6select21EpollSelectDispatcherQx9eventLoopMFMDFZbMDFC9ExceptionZbZv (this=0x7ffa4bc7f700, 
    unhandled_exception_hook=0x0, __unhandled_exception_hook_8=0x0, select_cycle_hook=0x7ffa4bc7f600, 
    __select_cycle_hook_8=0xc16f60 <_D5ocean4task9SchedulerQk17select_cycle_hookMFZb>)
    at ./submodules/ocean/src/ocean/io/select/EpollSelectDispatcher.d:749
#8  0x0000000000c16c77 in _D5ocean4task9SchedulerQk9eventLoopMFZv (this=0x125f410 <_D6common4util4task8MainTaskQj6__vtblZ>)
    at ./submodules/ocean/src/ocean/task/Scheduler.d:562

gavin-norman-sociomantic commented 5 years ago

This is something that happens occasionally? Or did you change something that seems to trigger this?

daniel-zullo commented 5 years ago

It does not happen very often, the rate is roughly once every three months. No, I didn't change anything in the project

sociomantic-tsunami / dlsproto

Problem disconnecting a GetRange request #102