Closed matthewmueller closed 4 years ago
Great to hear that! I believe it shouldn't be a problem running QuickJS inside http.Handler
's, since net/http
spawns up unique goroutines per request.
Nonetheless, runtime.LockOSThread()
will cause a bit of a performance impact; though it should be negligible.
Thanks so much for your response. I wrote a quick little program to benchmark it and performance seems quite great.
package main
import (
"fmt"
"net/http"
"strconv"
"time"
"github.com/lithdew/quickjs"
)
func main() {
http.ListenAndServe(":3000", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
one := time.Now()
js := quickjs.NewRuntime()
defer js.Free()
fmt.Println(time.Since(one))
two := time.Now()
context := js.NewContext()
defer context.Free()
fmt.Println(time.Since(two))
three := time.Now()
result, err := context.Eval(`1 + 2 * 100 * Math.random()`)
if err != nil {
fmt.Println(err)
return
}
defer result.Free()
fmt.Println(time.Since(three))
w.Write([]byte(strconv.Itoa(int(result.Int64()))))
}))
}
Happily churns through 500/req/sec 😬
echo "GET http://localhost:3000/" | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests [total, rate, throughput] 15000, 500.04, 500.03
Duration [total, attack, wait] 29.998s, 29.998s, 566.909µs
Latencies [min, mean, 50, 90, 95, 99, max] 459.406µs, 666.774µs, 615.891µs, 869.536µs, 986.747µs, 1.035ms, 13.698ms
Bytes In [total, mean] 36938, 2.46
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:15000
Error Set:
I was a bit surprised that I needed to stick quickjs.NewRuntime()
in the handler itself. Without it, I end up with an error (that's empty) in result, err := context.Eval("1 + 2 * 100 * Math.random()")
after the 2nd request. Happy to open a more specific issue if you see this as a potential bug.
Also since I'm not too familiar with runtime.LockOSThread()
, I wanted to try and produce a bad outcome by omitting it. So far I haven't been able to produce anything. Is the problem that is a race condition between goroutines trying to edit memory in C? Any pointers on how I can witness the runtime.LockOSThread()
issue?
The problem is that QuickJS allocates memory in the thread it's initialized within. As goroutines get scheduled and moved around to different threads, this means that a goroutine holding an instance to a quickjs.Runtime
may all of a sudden realize the memory it was holding would not exist in its current running thread, causing a segmentation fault.
Hence why runtime.LockOSThread
is necessary.
The issue is demonstrated in this test that was made here which then led to Guideline 5 being established: https://github.com/lithdew/quickjs/blob/master/quickjs_test.go#L130
Removing runtime.LockOSThread()
in the test above will cause a segmentation fault.
Thinking about it, by the way, typically it is advised against creating a single runtime per HTTP request handled (doing so would cause possibly hundreds of runtimes being created, which are expensive to initialize).
What could be done instead is creating a worker pool of runtime.NumCPUs()
goroutines locked to runtime.NumCPUs()
threads with their own unique quickjs.Runtime
instances. Your http.Handler
's can then submit work to them to be processed.
Ah that makes more sense. I'll give that test a whirl! I'll also bench the worker pattern and post my results here. From what I saw it was actually creating the context that took the most time:
Create Runtime: 35.067µs
Create Context: 236.605µs
Eval: 31.383µs
Thanks for taking the time to share your knowledge!
Hey there! I wanted to check-in after I did some additional research. I was able to get pooling working with the following code:
Overall, it's working quite fast, but I tested it with some CPU intensive tasks and I noticed that by using runtime.LockOSThread()
, the goroutines may get stuck on the same thread. You lose a lot of concurrency if this is the case.
I was wondering:
Thanks!
Hm, the only way you'd be able to circumvent goroutines being stuck on the same thread is by manually setting the goroutine's thread affinity using cgo.
http://pythonwise.blogspot.com/2019/03/cpu-affinity-in-go.html
The alternative is to also manually manage a thread pool over cgo.
Thanks for putting this package together! It works great out of the box without any additional installation scripts.
I was wondering about Guideline 5.
Does this make QuickJS unsuitable to run inside
http.Handler
s where you can have many goroutines servicing requests at the same time? The use-case in mind is server-side rendering.