search

forkbreak / fork_break

Fork with breakpoints, for testing multiprocess behaviour.
MIT License
61 stars 6 forks source link

Raise exception through breakpoints IO pipe #18

Closed biinari closed 4 years ago

biinari commented 6 years ago

Re-use breakpoints pipe to send exceptions through to parent. Works around issue with blocking on IO with no reader ready on the Fork ctrl pipe.

pedrocarrico commented 6 years ago

Hi @biinari

Thanks for your contribution, I'll review this shortly.

Do you mind sharing the motivation for this change and also your use case? A simple code example would be great.

Thanks!

biinari commented 6 years ago

Hi @pedrocarrico, actually this solution doesn't seem to fix my use case properly. I'll run up a minimal code example when I get a moment.

Essentially, we're running two processes concurrently to test for any issues in our code using database locks. The problem seems to occur when an exception with a large stack trace gets raised. fork's exception handling mechanism which writes to the ctrl pipe seems to block on write IO in the child process while fork_break parent is blocking on reading the from_fork pipe.

I have tried debugging by altering Fork.write_marshalled to use non_blocking writes (with retry) and found it writes around 1400 bytes ok before blocking, but in the case I was testing, the marshalled exception was around 18000 bytes.

pedrocarrico commented 6 years ago

@biinari I wonder if it's related to https://github.com/forkbreak/fork_break/issues/16 🤔

If you make a test reproducing the issue I would be glad to help you out debug and find a way to fix it. We can also screen share and you can show me if you'd like.

Many thanks!

pedrocarrico commented 6 years ago

@biinari Any news about this?