golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.79k stars 17.64k forks source link

[windows 10] cgo thread hangs forever #68559

Closed CascadingRadium closed 3 months ago

CascadingRadium commented 3 months ago

Go version

go version 1.22.5

Output of go env in your module/workspace:

cannot get go env as issue is seen in production with a built binary

What did you do?

Call go-faiss Train API using 2 parallel go routine on windows 10. (multithreaded openBLAS library + faiss lib)

What did you see happen?

Highly Reproducible issue

Basically we build openBLAS in multithreaded mode using USE_THREAD=1 to make it fast and then we build FAISS to use this multithreaded openBLAS. We use FAISS for doing vector search. Since Faiss is a C++ library we use a wrapper library called go-faiss that provided cgo bindings to use. There is a CGO api called as Train() that trains a faiss index. When 2 go routines call the Train API at the same time one of them never returns and ends up looping forever.

Basically we see that the 2 go routines spawn multiple windows threads in their specific openBLAS call as part of the Train() API, and these windows threads never return leading to infinite poll

Screenshot 2024-07-24 at 2 33 31 AM

reference https://github.com/blevesearch/go-faiss/blob/8a80e00275e2ab4b8af601d045e8d4343c2bcb40/index.go#L109

What did you expect to see?

The thread should not get stuck

seankhliao commented 3 months ago

This doesn't look actionable from the Go side, and seems more like a library issue.