Encoding the same text twice gives different results.

[JackKCWong]

When I try to run the textencoding example with the default sentence-transformers/all-MiniLM-L6-v2, it gives different results on the same text on 2 inferences. e.g. the r1.Vector.Sum() is different from r2.Vector.Sum() even though their first 10 elements are the same. And slices.Equal returns false.

They even output different results between go run.

    fn := func(text string) textencoding.Response {
        result, err := m.Encode(context.Background(), text, int(bert.MeanPooling))
        if err != nil {
            panic(err)
        }
        // fmt.Println(result.Vector.Data().F64()[:limit])
        return result
    }
    r1 := fn("see you later")
    fmt.Println(r1.Vector.Data().F64()[:limit])
    r2 := fn("see you later")
    fmt.Println(r2.Vector.Data().F64()[:limit])

    fmt.Println(r1.Vector.Sum())
    fmt.Println(r2.Vector.Sum())
    fmt.Println(slices.Equal(r1.Vector.Data().F32(), r2.Vector.Data().F32()))
    fmt.Println(Cosine(r1.Vector.Data().F64(), r2.Vector.Data().F64()))

[matteo-grella]

What is the architecture? amd64 o arm64?

[JackKCWong]

Apple M1

[matteo-grella]

Did you build and execute using the environment variable GOARCH=amd64?

[JackKCWong]

No, just using default settings. I just tried on a amd64 VM, also seeing the same behavior.

I am using these versions:

    github.com/nlpodyssey/cybertron v0.1.2 // indirect
    github.com/nlpodyssey/gopickle v0.1.0 // indirect
    github.com/nlpodyssey/gotokenizers v0.2.0 // indirect
    github.com/nlpodyssey/spago v1.0.1 // indirect
    github.com/nlpodyssey/spago/embeddings/store/diskstore v0.0.0-20220801114813-013d65be77fb // indirect

[JackKCWong]

My code here: https://github.com/JackKCWong/go-miniLM/blob/main/examples/exam.go

[matteo-grella]

Try to delete the downloaded model and redo again from scratch using GOARCH=amd64 before the building command and before the go run.

This is to ensure that the conversion from PyTorch uses the right float precision.

The library is optimized to run in x86-64 CPUs. If you want to run it on a different architecture, you can use the GOARCH=amd64 environment variable.

I think you are encountering a float precision issue on arm operations. I’m using an Apple M1 as well and always executing with envvar.

[JackKCWong]

I tried GOARCH=amd64 on M1, doesn't seem to make any difference. It seems to have sth to do with the textencoding.Response being returned from a func. e.g. I expect infer1 and infer2 to give the same results, but infer2 just give random results each time while infer1 gives a correct result (which I know from comparing to python) The only difference is infer2 wraps the inference in a func that returns a *textencoding.Response.

func infer2(m textencoding.Interface) {
    fn := func(text string) *textencoding.Response {
        result, err := m.Encode(context.Background(), text, int(bert.MeanPooling))
        if err != nil {
            panic(err)
        }
        return &result
    }
    r1 := fn("see you tomorrow")
    r2 := fn("see you later")
    fmt.Println(Cosine(r1.Vector.Data().F64(), r2.Vector.Data().F64()))
}

func infer1(m textencoding.Interface) {
    r1, err := m.Encode(context.Background(), "see you tomorrow", int(bert.MeanPooling))
    if err != nil {
        log.Fatal().Err(err).Send()
    }
    f1 := r1.Vector.Data().F64()
    r2, err := m.Encode(context.Background(), "see you later", int(bert.MeanPooling))
    if err != nil {
        log.Fatal().Err(err).Send()
    }
    f2 := r2.Vector.Data().F64()
    fmt.Println(Cosine(f1, f2))
}

full code: https://github.com/JackKCWong/go-miniLM/blob/main/main.go

[matteo-grella]

Thanks! I'll fix the bug in the next few hours. Ping you here.

Anyway, always use GOARCH=amd64. It gives you 100x speed up.

[matteo-grella]

The commit ef2ca134a6d388fe9cc0bcad1bbf45108aaf13cf solved the issue.

Import the Cybertron version v0.1.3-0.20230219111654-ef2ca134a6d3 in your project.

Let me know and thank you for spotting this bug!

[JackKCWong]

Works like a charm :)

Thanks for the quick turnaround! Can't wait to take it out for a walk in the wild.