Closed neclepsio closed 6 years ago
ALTree
commented
6 years ago This kind of errors are almost always caused by bad cgo code or by erroneous (racy) Go code. Usual questions:
neclepsio
commented
6 years ago @ALTree Thank you.
ALTree
commented
6 years ago the fact it does not crash under go 1.7.6 is maybe relevant to exclude problems in this field?
Unfortunately, I don't think it is. This kind of crashes are often not completely deterministic, so it's possible that a program that incorrectly uses cgo gets away with it in go1.7 by sheer luck, and then starts crashing in later releases due to changes in the runtime's internal implementation details, or other similar factors.
Considering you can't share the code and we don't have a reproducer, I don't think there's much that can be done here to investigate the issue or help you.
My guess is that the crash is induced by memory corruption caused by the cgo part of your program. If you could provide a race-free, cgo-free reproducer (like the one we had in #22781) then someone will certainly look into it, otherwise this issue is not really actionable.
RLH
commented
6 years ago A bit of a background might help. This check was added in order to detect accounting errors, in this case the GC has found more reachable objects than expected. This is typically caused when a pointer is hidden from the GC either due to a race condition, unsafe manipulations, or CGO hackery. The pointer is later resurrected and noticed by the runtime. The fact that no error was reported in earlier versions of Go is not an indication that this wasn't always a bug, it's just that the runtime just started flagging it.
Of course the bug may not be in the application at all but in the runtime or in the compiler. Unfortunately, since the problem happened far away from where the runtime found the accounting error a reproducer is probably needed to determine whether the bug is in the runtime or compiler.
Setting GOGC to a low number will induce many GCs and hopefully increase the chances that the bug will trigger. I have used this technique to decrease non-determinism and help create reproducers.
neclepsio
commented
6 years ago Thank you. In an attempt to make a cgo-free reproducer (which I'm far from), I discovered something that is maybe useful. I had this code:
ok := make(chan int)
go func() {
// clear output folder
ok <- 1
}()
go func() {
// read 3d models and add to result channel
ok <- 1
}()
go func() {
// read text file
<- ok
<- ok
// read from result channel and queue adding models to the view in OpenGL thread
}()Just changing it to use a sync.WaitGroup instead of ok made the issue happen much less. From 40% of times (with a particular set of data), it now happens just 10% of times. There were no other changes in the code. I can also confirm that in all my tests I got no warning from race detector.
After that, I added a "rate limiter" to the goroutine adding models to the view (which runs in the OpenGL thread): before, it added the models as soon as they where sent to the (unbuffered) channel. Now the channel is buffered and the goroutine only adds the models to the view once per second. After that, I got no crashes.
neclepsio
commented
6 years ago @RLH thank you for your explaination, which I read after my message (which waited to be confirmed for several hours while I did my tests). You are right, after I changed GOGC the issue resurfaced. Since we suspect the issue is in cgo library, I don't know if the issue should be reopen or kept closed until I can write a reproducer.
ALTree
commented
6 years ago Since we suspect the issue is in cgo library, I don't know if the issue should be reopen or kept closed until I can write a reproducer.
Let's keep this closed, and open a new one when (if) you find a reasonably short reproducer.
What version of Go are you using (
go version)?go version go1.10.1 windows/amd64Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env)?What did you do?
I'm developing an opengl program using go-qml; when loading complex models the runtime it often crashes using version 1.10.1,1.10, 1.9.5 (in which a similar issue, #22781, has been fixed), 1.8.7.
It seems to work in 1.7.6.
What did you expect to see?
No error.
What did you see instead?
A crash:
Another:
Please help me sharing all the data you need; I have not been able write a reproducer and I cannot share source or models.