Is there a way to use this with TensorFlow?

gpujs / gpu.js

GPU Accelerated JavaScript

https://gpu.rocks

MIT License

15.1k stars 654 forks source link

Is there a way to use this with TensorFlow? #627

Closed NullVoxPopuli closed 4 years ago

NullVoxPopuli commented 4 years ago

https://i.giphy.com/KeVooDIaiNKXCx7snR.gif (idk how to add gif previews on GH)

Hello!

I just have a couple questions:

is gpu.js compat with TensorFlow? What would that look like?
would there be any benefit to gpu.js being WASM-backed?
does gpu.js work in a Web Worker?

I'm working on a web app for exploring reinforcement learning, and have run in to a js-perf bottleneck. I've moved my A.I. code to a web worker so at least it doesn't block the main thread -- so the user can still click on things / change things / etc.

But even in a web worker, JS seems too slow -- at least on my CPUs anyway.

My Project is over here: https://github.com/NullVoxPopuli/doctor-who-thirteen-game-ai

Thanks!

harshkhandeparkar commented 4 years ago

Tensorflow is already GPU accelerated :) WASM - I don't think so, not much benefit other than faster compile times Web worker - It should but I am not sure.

NullVoxPopuli commented 4 years ago

My situation seems a bit unexplored

TensorFlow works in web workers
but TensorFlow can't save models from web workers, so I need to figure out how to serialize to/from the main thread for saving
I've read that TensorFlow uses SVG for GPU acceleration in the browser
there is no SVG in web workers
WASM is much faster than JS 🤷 (I know that there is an optional WASM backend for TensorFlow.js, but idk yet how to check if I'm using that)
I'd like to parallelize reinforcement learning -- like, play parallel games on different web workers than merge the results for the next learning iteration

I know GPUs are fast and good at parallel things, but I don't know how they'd play in to the above 🤷

harshkhandeparkar commented 4 years ago

GPU.js just compiles the js into GLSL, a language that can be interpreted by a GPU. It uses the <canvas> element to interface with the GPU. (It does work on converting and sending/receiving data from a GPU)

I don't it will be benefitted much by wasm. It will work faster only if there is a big kernel to be built but kernel building time hardly matters.

Web workers are weird.

NullVoxPopuli commented 4 years ago

cool! thanks for explaining.

Also, after some additional research, I think I've found that TensorFlow can use a WebGL (and soon WebGPU?) backend, https://blog.logrocket.com/ai-in-browsers-comparing-tensorflow-onnx-and-webdnn-for-image-classification/

So I think for what I need, I don't need gpu.js.

thanks though! <3