TFJS basic operations are extremely slow in comparison to Native JS

botcs commented 1 month ago

Issue type

Performance

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

tfjs

Custom code

No

OS platform and distribution

No response

Mobile device

No response

Python version

No response

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

I was just shocked when I saw that even basic operations like passing an array to tf.tensor is 60% slower than allocating a tf.buffer, setting each value one by one and calling toTensor(). When it comes to tf.randomNormal, which I believe is called many many times, there is a major slow down, even though the same Box-Muller algorithm is used.

Standalone code to reproduce the issue

<!DOCTYPE html>
<html>
<head>
    <title>Random Number Benchmark</title>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
</head>
<body>
    <script>
        async function benchmarkTfjsRandomNormal(numSamples) {
            const start = performance.now();

            // Generate random floats using TensorFlow.js
            const randomFloats = tf.randomNormal([numSamples]);

            // Ensure the tensors are evaluated
            await randomFloats.data();

            const end = performance.now();
            tf.dispose([randomFloats]); // Clean up tensors
            return end - start;
        }

        function generateBoxMullerNormal() {
            let u = 0, v = 0;
            while (u === 0) u = Math.random(); // Converting [0,1) to (0,1)
            while (v === 0) v = Math.random();
            return Math.sqrt(-2.0 * Math.log(u)) * Math.cos(2.0 * Math.PI * v);
        }

        function benchmarkNativeRandomNormal(numSamples) {
            const start = performance.now();

            const randomFloats = new Float32Array(numSamples);
            for (let i = 0; i < numSamples; i++) {
                randomFloats[i] = generateBoxMullerNormal();
            }

            const end = performance.now();
            return end - start;
        }

        async function runBenchmarks() {
            const sampleSizes = [1000, 10000, 100000, 1000000];
            for (const numSamples of sampleSizes) {
                const tfjsTime = await benchmarkTfjsRandomNormal(numSamples);
                console.log(`TensorFlow.js (${numSamples} samples): ${tfjsTime.toFixed(2)} ms`);

                const nativeTime = benchmarkNativeRandomNormal(numSamples);
                console.log(`Native JavaScript (${numSamples} samples): ${nativeTime.toFixed(2)} ms`);

                const speedup = nativeTime / tfjsTime;
                console.log(`Speedup: ${speedup}`);
                console.log('');
            }
        }

        // Run the benchmarks and log the results
        runBenchmarks();
    </script>
</body>
</html>

Relevant log output

TensorFlow.js (1000 samples): 11.00 ms
test.html:49 Native JavaScript (1000 samples): 0.40 ms
test.html:52 Speedup: 0.03636363690549677
test.html:53 
test.html:46 TensorFlow.js (10000 samples): 2.40 ms
test.html:49 Native JavaScript (10000 samples): 0.90 ms
test.html:52 Speedup: 0.3750000015522043
test.html:53 
test.html:46 TensorFlow.js (100000 samples): 5.90 ms
test.html:49 Native JavaScript (100000 samples): 5.20 ms
test.html:52 Speedup: 0.8813559292925052
test.html:53 
test.html:46 TensorFlow.js (1000000 samples): 64.80 ms
test.html:49 Native JavaScript (1000000 samples): 58.20 ms
test.html:52 Speedup: 0.8981481482120248
test.html:53 
test.html:46 TensorFlow.js (10000000 samples): 648.90 ms
test.html:49 Native JavaScript (10000000 samples): 591.00 ms
test.html:52 Speedup: 0.9107720758122536

botcs commented 1 month ago

I am not sure if warmup needed for these experiments

sushreebarsa commented 3 weeks ago

@botcs TensorFlow.js uses the WebGL backend for faster computations. Please ensure WebGL is enabled and utilized. For any further queries please raise a ticket in TFJS repository. Thank you!

github-actions[bot] commented 2 weeks ago

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

github-actions[bot] commented 1 week ago

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

google-ml-butler[bot] commented 1 week ago

Are you satisfied with the resolution of your issue? Yes No

tensorflow / tensorflow