Possible to use inside an http.Handler?

[matthewmueller]

Thanks for putting this package together! It works great out of the box without any additional installation scripts.

I was wondering about Guideline 5.

Make sure to call runtime.LockOSThread() to ensure that QuickJS always operates in the exact same thread.

Does this make QuickJS unsuitable to run inside http.Handlers where you can have many goroutines servicing requests at the same time? The use-case in mind is server-side rendering.

[lithdew]

Great to hear that! I believe it shouldn't be a problem running QuickJS inside http.Handler's, since net/http spawns up unique goroutines per request.

Nonetheless, runtime.LockOSThread() will cause a bit of a performance impact; though it should be negligible.

[matthewmueller]

Thanks so much for your response. I wrote a quick little program to benchmark it and performance seems quite great.

package main

import (
    "fmt"
    "net/http"
    "strconv"
    "time"

    "github.com/lithdew/quickjs"
)

func main() {
    http.ListenAndServe(":3000", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        one := time.Now()
        js := quickjs.NewRuntime()
        defer js.Free()
        fmt.Println(time.Since(one))
        two := time.Now()
        context := js.NewContext()
        defer context.Free()
        fmt.Println(time.Since(two))

        three := time.Now()
        result, err := context.Eval(`1 + 2 * 100 * Math.random()`)
        if err != nil {
            fmt.Println(err)
            return
        }
        defer result.Free()
        fmt.Println(time.Since(three))

        w.Write([]byte(strconv.Itoa(int(result.Int64()))))
    }))
}

Happily churns through 500/req/sec 😬

echo "GET http://localhost:3000/" | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests      [total, rate, throughput]         15000, 500.04, 500.03
Duration      [total, attack, wait]             29.998s, 29.998s, 566.909µs
Latencies     [min, mean, 50, 90, 95, 99, max]  459.406µs, 666.774µs, 615.891µs, 869.536µs, 986.747µs, 1.035ms, 13.698ms
Bytes In      [total, mean]                     36938, 2.46
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:15000
Error Set:

I was a bit surprised that I needed to stick quickjs.NewRuntime() in the handler itself. Without it, I end up with an error (that's empty) in result, err := context.Eval("1 + 2 * 100 * Math.random()") after the 2nd request. Happy to open a more specific issue if you see this as a potential bug.

Also since I'm not too familiar with runtime.LockOSThread(), I wanted to try and produce a bad outcome by omitting it. So far I haven't been able to produce anything. Is the problem that is a race condition between goroutines trying to edit memory in C? Any pointers on how I can witness the runtime.LockOSThread() issue?

[lithdew]

The problem is that QuickJS allocates memory in the thread it's initialized within. As goroutines get scheduled and moved around to different threads, this means that a goroutine holding an instance to a quickjs.Runtime may all of a sudden realize the memory it was holding would not exist in its current running thread, causing a segmentation fault.

Hence why runtime.LockOSThread is necessary.

The issue is demonstrated in this test that was made here which then led to Guideline 5 being established: https://github.com/lithdew/quickjs/blob/master/quickjs_test.go#L130

Removing runtime.LockOSThread() in the test above will cause a segmentation fault.

Thinking about it, by the way, typically it is advised against creating a single runtime per HTTP request handled (doing so would cause possibly hundreds of runtimes being created, which are expensive to initialize).

What could be done instead is creating a worker pool of runtime.NumCPUs() goroutines locked to runtime.NumCPUs() threads with their own unique quickjs.Runtime instances. Your http.Handler's can then submit work to them to be processed.

[matthewmueller]

Ah that makes more sense. I'll give that test a whirl! I'll also bench the worker pattern and post my results here. From what I saw it was actually creating the context that took the most time:

Create Runtime: 35.067µs
Create Context: 236.605µs
Eval: 31.383µs

Thanks for taking the time to share your knowledge!

[matthewmueller]

Hey there! I wanted to check-in after I did some additional research. I was able to get pooling working with the following code:

quickjs.go

```go package quickjs import ( "context" "fmt" "runtime" "github.com/lithdew/quickjs" ) // NewWorker creates a new worker // Worker is not goroutine safe func NewWorker() *Worker { runtime := quickjs.NewRuntime() return &Worker{runtime} } // Worker struct type Worker struct { runtime quickjs.Runtime } // Eval function func (w *Worker) Eval(ctx context.Context, js string, v interface{}) (err error) { context := w.runtime.NewContext() defer context.Free() result, err := context.Eval(js) if err != nil { return err } switch t := v.(type) { case *string: // we expect a string if !result.IsString() { return fmt.Errorf("Eval expected a string, but received %q", result) } *t = result.String() result.Free() return nil case *int64: if !result.IsNumber() { return fmt.Errorf("Eval expected a number, but received %q", result) } *t = result.Int64() result.Free() return nil case *bool: if !result.IsBool() { return fmt.Errorf("Eval expected a boolean, but received %q", result) } *t = result.Bool() result.Free() return nil case nil: // ignore the output result.Free() return nil default: result.Free() return fmt.Errorf("quickjs.Eval: unable to coerce into expected value %t", t) } } // Close the worker func (w *Worker) Close() { w.runtime.Free() } // NewPool creates a pool of workers func NewPool(size int) *Pool { requestCh := make(chan request, size) pool := &Pool{requestCh} // start all available workers for i := 0; i < size; i++ { go pool.worker(i) } return pool } // Pool of workers type Pool struct { requestCh chan request } // TODO: sometimes goroutines get scheduled on the same thread. // then they're locked to that thread. You can see this with // -trace. Is there anyway to force the goroutines to get // scheduled on different threads? // // Even better, is it possible to remove this requirement // entirely? By cleaning up state after each run or something? func (p *Pool) worker(id int) { runtime.LockOSThread() defer runtime.UnlockOSThread() w := NewWorker() defer w.Close() for req := range p.requestCh { err := w.Eval(req.ctx, req.js, req.result) req.response <- response{err} } } type request struct { ctx context.Context js string result interface{} response chan response } type response struct { err error } // Eval the worker func (p *Pool) Eval(ctx context.Context, js string, result interface{}) (err error) { response := make(chan response) p.requestCh <- request{ctx, js, result, response} res := <-response return res.err } // Close the pool func (p *Pool) Close() { close(p.requestCh) } ```

quickjs_test.go

```go package quickjs_test import ( "context" "fmt" "runtime" "testing" "time" "github.com/matthewmueller/duo/internal/quickjs" "github.com/tj/assert" "golang.org/x/sync/errgroup" ) const slow = ` function slow(baseNumber){ let result = 0; for (var i = Math.pow(10, 7); i >= 0; i--) { result += Math.atan(i) * Math.tan(i); }; return Math.floor(result) } slow(10) ` func TestWorker(t *testing.T) { ctx := context.Background() var eg errgroup.Group eg.Go(func() error { worker := quickjs.NewWorker() defer worker.Close() var result int64 now := time.Now() err := worker.Eval(ctx, slow, &result) assert.NoError(t, err) fmt.Println(result) fmt.Println(time.Since(now)) var result2 int64 now2 := time.Now() err2 := worker.Eval(ctx, slow, &result2) assert.NoError(t, err2) fmt.Println(time.Since(now2)) return nil }) eg.Wait() // assert.Equal(t, "11", v) } const slow = ` function slow(baseNumber){ let result = 0; for (var i = Math.pow(10, 7); i >= 0; i--) { result += Math.atan(i) * Math.tan(i); }; return Math.floor(result) } slow(10) ` func TestPool(t *testing.T) { ctx := context.Background() pool := quickjs.NewPool(runtime.NumCPU()) defer pool.Close() var eg errgroup.Group for i := 0; i < 204; i++ { eg.Go(func() error { var result int64 err := pool.Eval(ctx, slow, &result) assert.NoError(t, err) assert.Equal(t, int64(-2898551), result) return nil }) } assert.NoError(t, eg.Wait()) } ```

Overall, it's working quite fast, but I tested it with some CPU intensive tasks and I noticed that by using runtime.LockOSThread(), the goroutines may get stuck on the same thread. You lose a lot of concurrency if this is the case.

I was wondering:

1. Is it possible to remove this requirement? Perhaps by cleaning after each run?
2. Do you know how to schedule goroutines on a specific thread or tell them to run on different threads?
3. Any other workarounds you can think of to get uniform parallelism?

Thanks!

[lithdew]

Hm, the only way you'd be able to circumvent goroutines being stuck on the same thread is by manually setting the goroutine's thread affinity using cgo.

http://pythonwise.blogspot.com/2019/03/cpu-affinity-in-go.html

The alternative is to also manually manage a thread pool over cgo.