simsong / bulk_extractor

This is the development tree. Production downloads are at:
https://github.com/simsong/bulk_extractor/releases
Other
1.08k stars 185 forks source link

Deadlock on 1e211e01fb65479434b78211dc3a816ffe79fdd4 when run with libprofiler #297

Closed jonstewart closed 2 years ago

jonstewart commented 2 years ago

I've twice gotten deadlock running the above commit (currently HEAD on main) with the ubnist1.E01 image.

I was using the google profiler, so I could see that affecting the normal timing of when threads grab specific locks. But it shouldn't change the correctness, so there must be some race that allows this to occur. Attached is stack trace output from gdb for all the threads on one of these runs (I just did a default run):

gdb.txt

simsong commented 2 years ago

Thanks. I’m not getting a deadlock, and I did add some cool stuff, so it’s possible. What OS are you running on?


Sent from my phone.

On Nov 28, 2021, at 4:00 PM, Jon Stewart @.***> wrote:

 I've twice gotten deadlock running the above commit (currently HEAD on main) with the ubnist1.E01 image.

I was using the google profiler, so I could see that affecting the normal timing of when threads grab specific locks. But it shouldn't change the correctness, so there must be some race that allows this to occur. Attached is stack trace output from gdb for all the threads on one of these runs (I just did a default run):

gdb.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

simsong commented 2 years ago

Oh, do you get a deadlock every time, or just sometimes?


Sent from my phone.

On Nov 28, 2021, at 4:00 PM, Jon Stewart @.***> wrote:

 I've twice gotten deadlock running the above commit (currently HEAD on main) with the ubnist1.E01 image.

I was using the google profiler, so I could see that affecting the normal timing of when threads grab specific locks. But it shouldn't change the correctness, so there must be some race that allows this to occur. Attached is stack trace output from gdb for all the threads on one of these runs (I just did a default run):

gdb.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

simsong commented 2 years ago

Also, I think that your profiler isn’t going to work as well as the built-in profiler


Sent from my phone.

On Nov 28, 2021, at 4:00 PM, Jon Stewart @.***> wrote:

 I've twice gotten deadlock running the above commit (currently HEAD on main) with the ubnist1.E01 image.

I was using the google profiler, so I could see that affecting the normal timing of when threads grab specific locks. But it shouldn't change the correctness, so there must be some race that allows this to occur. Attached is stack trace output from gdb for all the threads on one of these runs (I just did a default run):

gdb.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

simsong commented 2 years ago

The locks are in libprofiler. That is the source of the problem.


Sent from my phone.

On Nov 28, 2021, at 4:00 PM, Jon Stewart @.***> wrote:

 I've twice gotten deadlock running the above commit (currently HEAD on main) with the ubnist1.E01 image.

I was using the google profiler, so I could see that affecting the normal timing of when threads grab specific locks. But it shouldn't change the correctness, so there must be some race that allows this to occur. Attached is stack trace output from gdb for all the threads on one of these runs (I just did a default run):

gdb.txt

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

simsong commented 2 years ago

Please try to replicate without libprofiler and reopen if you can.