High latency even for the smallest operation

[hssngl]

thanks for this great package. I have some difficulties in terms of latency, is this normal in the context of gpu.js?

[harshkhandeparkar]

The latency is probably because GPU.js compiles the js into GLSL initially. This is done only once so you won't face it if you use the same kernel again.

const kernel = new GPU.createKernel(...);
kernel(inputs) // latency
kernel(inputs) // no more

[robertleeplummerjr]

Can you reproduce the latency you are seeing using an example that runs in either the browser (jsfiddle or observablehq) or node? The initial run will have latency, as it is expensive because it compiles javascript (typeless) to GLSL (stongly typed). Subsequent calls to the original kernel where alternate types that are compatible are used (for example, using a texture from pipeline mode, vs an array) will cause a recompilation as well.

[benrayfield]

This attached html (and dependency js) file, used with gpu-browser.min.js in brave, chrome, and firefox, in win10pro, with a Nvidia Geforce RTX 2080 SUPER GPU, gets these dt=seconds times for reusing same kernel on matrix multiply of 512x512 by 512x512 (counting 512^3 flops, though you might want to double that if multiply-then-add counts as 2 flops):

i=1 dt=0.45799994468688965 gflops=0.134217728 gflopsPerSec=0.2930518432524213 i=2 dt=0.012000083923339844 gflops=0.134217728 gflopsPerSec=11.184732444991496 i=3 dt=0.01399993896484375 gflops=0.134217728 gflopsPerSec=9.587022367529157 i=4 dt=0.00800013542175293 gflops=0.134217728 gflopsPerSec=16.776932004807392 i=5 dt=0.006999969482421875 gflops=0.134217728 gflopsPerSec=19.174044735058313 i=6 dt=0.006999969482421875 gflops=0.134217728 gflopsPerSec=19.174044735058313

testGpujsLatency_512x512_multipleCallsOfSameKernel.html.zip