josephrocca
commented
1 year ago Looks like the same issue: https://twitter.com/praeclarum/status/1659490569633280001 - just related to precision thresholds.
Open replete opened 1 year ago
josephrocca
commented
1 year ago Looks like the same issue: https://twitter.com/praeclarum/status/1659490569633280001 - just related to precision thresholds.
xenova
commented
1 year ago The only real issue is xlogy which returns 0; all other's are just precision issues.
praeclarum
commented
1 year ago The one that's bothering me (yeah @xenova, xlogy has been driving me nuts) is the pow([-0.5], [-0.5]). It's interesting that this hardware returns sqrt(2). It should be 2i*sqrt(2)
The hardware is ignoring the imaginary part but keeping the sort(2). My hardware (ATI on an iMac) returns 0. As does PyTorch. But this hardware is interesting in that it's keeping some of the values.
Fortunately, this is pretty edge-case stuff and shouldn't affect real nets. But it certainly is interesting.
xenova
commented
1 year ago The one that's bothering me (yeah @xenova,
xlogyhas been driving me nuts) is thepow([-0.5], [-0.5]). It's interesting that this hardware returnssqrt(2). It should be2i*sqrt(2)
Surely that's just because it's float and not complex?
>>> import torch
>>> torch.tensor(-0.5).pow(-0.5)
tensor(nan)
>>> torch.tensor(-0.5).to(torch.complex64).pow(-0.5)
tensor(0.-1.4142j)Yeah. It shouldn't be returning any number at all since the GPU doesn't do complex natively. The WebGPU spec says all NaNs should be converted to 0 (flush to zero they call it):
https://www.w3.org/TR/WGSL/#floating-point-evaluation
@replete Would you let me know what your Mac hardware is? I'm not too concerned but it would be nice to know which models are doing this.
twodawg
commented
1 year ago pow([0], [0]) gradient | Expected «-Infinity» to be close to «0» (diff: < 0.0000005)Error: Expected «-Infinity» to be close to «0» (diff: < 0.0000005)at Expect.toBeCloseTo (https://praeclarum.org/webgpu-torch/tests/testfw.js:9:152)at compareArrays (https://praeclarum.org/webgpu-torch/tests/tests.js:1:415297)at https://praeclarum.org/webgpu-torch/tests/tests.js:1:416489at step (https://praeclarum.org/webgpu-torch/tests/tests.js:1:414312)at Object.next (https://praeclarum.org/webgpu-torch/tests/tests.js:1:414417)at fulfilled (https://praeclarum.org/webgpu-torch/tests/tests.js:1:413045) xlogy([0], [0.30000001192092896]) gradient | Expected «0» to be close to «-1.2039728164672852» (diff: < 0.0000005)Error: Expected «0» to be close to «-1.2039728164672852» (diff: < 0.0000005)at Expect.toBeCloseTo (https://praeclarum.org/webgpu-torch/tests/testfw.js:9:152)at compareArrays (https://praeclarum.org/webgpu-torch/tests/tests.js:1:415297)at https://praeclarum.org/webgpu-torch/tests/tests.js:1:416489at step (https://praeclarum.org/webgpu-torch/tests/tests.js:1:414312)at Object.next (https://praeclarum.org/webgpu-torch/tests/tests.js:1:414417)at fulfilled (https://praeclarum.org/webgpu-torch/tests/tests.js:1:413045)
These errors are on Mac Mini M1, MacOS 13.1, Edge 113.0.1774.50
replete
commented
1 year ago Yeah. It shouldn't be returning any number at all since the GPU doesn't do complex natively. The WebGPU spec says all
NaNsshould be converted to0(flush to zero they call it):https://www.w3.org/TR/WGSL/#floating-point-evaluation
@replete Would you let me know what your Mac hardware is? I'm not too concerned but it would be nice to know which models are doing this.
MacBook Pro 13" 2020 with Intel i5-1038g7 2.0ghz 4 core Ice Lake '10th Gen' 15w, Intel® Iris® Plus Graphics (G7) with 64 'execution units'.
Ran on real Chrome 113.0.5672.126, same test results.
wgaylord
commented
9 months ago Interestingly I also got the same results for xlogy on Chrome Version 119.0.6045.160 (Official Build) (64-bit) running on Windows 10 on a Intel i5 8265U cpu (has integrated graphics). So I wonder if this is just Chrome not following the spec?
113.0.5672.63 (ungoogled chromium) MacOS Ventura 13.3.1:
Hope this helps.