Closed mycroes closed 6 months ago
@gfoidl Would you care to reflect on the possible solution? I think I might as well need to flush the JobPool
on Dispose()
(do Reader.TryRead()
until it returns false
) and perhaps check for disposed/throw in RentJobIdAsync
as well.
The most annoying issue here is that even though there's a default request timeout of 5s in read/write, that's implemented using CancellationToken
and the final await jobPool.GetRequest(jobId)
isn't affected by the cancellation, so I'm actually hitting this bug in production.
Would it help to remove Dispose
, but introduce DisposeAsync
where the async dispose can await outstanding request (w/ timeout).
I think the synchronous dispose makes a lot of troubles (as we see through various issues) -- and it gets sync-over-async here. My thinking is that async disposal could get rid of these troubles.
For the async disposal when a timout is hit, then proceed as you suggested above (and drain the JobPool
as well).
BTW: are you able to produce a minimal repro of the problem or is it timing-relavant (race condition)?
Would it help to remove
Dispose
, but introduceDisposeAsync
where the async dispose can await outstanding request (w/ timeout).
I think this will move complexity, not sure if it really helps though. For one it would complicate my use of the library, because right now I don't have an async path to call DisposeAsync
.
I think the synchronous dispose makes a lot of troubles (as we see through various issues) -- and it gets sync-over-async here. My thinking is that async disposal could get rid of these troubles.
I actually think it's fine. Async is nothing more than postponed Action
s as far as we're concerned here, which means I can just invoke the postponed Action
s on Dispose
as suggested above.
For the async disposal when a timout is hit, then proceed as you suggested above (and drain the
JobPool
as well).
As you say, this would still require the sync Dispose
after the timeout. That does mean we could implement both though, the DisposeAsync()
with timeout and Dispose()
that just kills any requests right away. I don't really care about waiting, I'm mostly only closing the connections on errors anyway, as was the case when this bug showed up (but apparently there was at least data in the buffer to process a single request).
BTW: are you able to produce a minimal repro of the problem or is it timing-relavant (race condition)?
In context to ConcurrentRequestExecutor
it's timing relevant, but a test around JobPool
could expose the underlying problem.
because right now I don't have an async path
Ah, then that doesn't make sense to have DisposeAsync. The sync Dispose I had in mind to just use internally, and don't expose it.
Then your suggestions from above seem fine.
Thanks for reviewing! I think it would be fine to expose both Dispose
and DisposeAsync
, the latter needs some thought to implement though. I guess I'll be working on this tonight.
Consider following stack trace:
The following must have happened:
PerformRequest
ConcurrentRequestExecutor
is disposedreceiveSignal.TryRelease()
returnsfalse
Sally7Exception.ThrowFailedToSignalReceiveDone()
is invokedIf there was only one request, this results in the caller receiving the exception from point 6.
Now assume there was already a caller waiting:
rec.Complete
is called forRequest
BRequest
A (no cancellation)receiveSignal.Dispose()
Request
A is never completedPossible solution:
JobPool.Dispose()
Dispose()
allRequest
s a.Request.Dispose() { _disposed = true; _continuation.Invoke(); }
b.Request.GetResult() { ObjectDisposedException.ThrowIf(_disposed, this); return ... }
ObjectDisposedException.ThrowIf
toConcurrentRequestExecutor
throw paths?Disadvantages?