WASM program does not respond when repeatedly creating big data slices

Mrooze-zeng commented 3 years ago

I am new to Go and want to use wasm to calculate the md5 of a file, but I found that I can only calculate a 45Mb file twice, and then it will be stopped on this line. dst := make([]byte, args[0].Get("length").Int())I have to refresh the browser to make it work again. Is the memory usage high or what is the reason? I do not know. Is there a way to release it?

I compiled the same code with Go's built-in WASM and it worked well.

My tinygo version: tinygo version 0.19.0 darwin/amd64 (using go version go1.16.5 and LLVM version 11.0.0)

Here is the my code:

main.go:

package main

import (
    "crypto/md5"
    "fmt"
    "syscall/js"
)

func getMD5() js.Func {
    return js.FuncOf(func(this js.Value, args []js.Value) interface{} {
        uint8Array := js.Global().Get("Uint8Array")
        uint8ClampedArray := js.Global().Get("Uint8ClampedArray")
        if len(args) < 1 || !(args[0].InstanceOf(uint8Array) || args[0].InstanceOf(uint8ClampedArray)) {
            return js.Undefined()
        }
        //stop on this line;
        dst := make([]byte, args[0].Get("length").Int())

        js.CopyBytesToGo(dst, args[0])
        if dst == nil {
            return js.Undefined()
        }
        return fmt.Sprintf("%x", md5.Sum(dst))
    })
}

func main() {
    js.Global().Set("getMd5", getMD5())
    select {}
}

index.html:

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8" />
  <title>Hello World - Go</title>
</head>
<body>
  <input
    type="file"
    id="j-file"
  >
  <button id="j-cal">btn</button>
  <script src="./wasm_exec.js"></script>
  <script
    type="module"
    src="./index.js"
  ></script>
</body>
</html>

index.js

(async function () {
  const go = new Go()
  let app
  if (WebAssembly.instantiateStreaming) {
    app = await WebAssembly.instantiateStreaming(fetch("app.wasm"), go.importObject)
  } else {
    const res = await fetch("app.wasm");
    app = await WebAssembly.instantiate(await res.arrayBuffer(), go.importObject)
  }
  const { instance } = app;
  go.run(instance)
  const $input = document.getElementById("j-file")
  const $btn = document.getElementById("j-cal")

  $btn.addEventListener("click", function () {
    const file = $input.files[0] || new Blob([]);
    const fileReader = new FileReader()
    fileReader.onload = function () {
      console.log((window.getMd5 || function () { })(new Uint8Array(this.result)))
    }
    fileReader.readAsArrayBuffer(file)
  })
})()

build command:

tinygo build -o ./demo/app.wasm -target wasm .

aykevl commented 3 years ago

What is the error that you get in the JavaScript console (developer tools)?

Mrooze-zeng commented 3 years ago

What is the error that you get in the JavaScript console (developer tools)?

There is no error, but the process is getting blocked.

Mrooze-zeng commented 3 years ago

What is the error that you get in the JavaScript console (developer tools)?

Sorry to copy and paste the code with issues, I have corrected it. But when I calculate the MD5 of a file larger than 45MB for the second time without refreshing the page, it is still blocked.

vanelizarov commented 3 years ago

@Mrooze-zeng have you found a solution? Currently experiencing same problem while repeatedly processing large files (~9MB)

codefromthecrypt commented 2 years ago

I might suggest using a tinygo //export function to do the MD5 instead of re-buffering. Since the memory is shared, you can use an offset/len pair to update the hasher. I think you can find some WebAssembly hash functions that do similarly even if their source isn't TinyGo. Ex. https://github.com/Daninet/hash-wasm

tinygo-org / tinygo

WASM program does not respond when repeatedly creating big data slices #1982