search

ashima / webgl-noise

Procedural Noise Shader Routines compatible with WebGL
MIT License
2.79k stars 302 forks source link

Problems in Safari iOS on mediump #25

Open gregtatum opened 3 years ago

gregtatum commented 3 years ago

It seems like snoise(vec3) and maybe others are only working using precision highp float; on Safari in iOS. I spent a bit of time debugging some older shaders of mine that aren't working anymore and this appeared to fix it.

gregtatum commented 3 years ago

This random codepen I found seems to exhibit the behavior: https://codepen.io/ykob/pen/qbwLaY

I'm not sure if it's the exact latest version of the shader in this repo.

Steps to reproduce:

stegu commented 3 years ago

Yes, high precision is required for the functions to work in WebGL. All functions require it, and it's not easy to fix, because the code that requires high precision is not the floating point computations, but the integers for the hashing. This is not a bug, it's simply a requirement for the algorithm to work. I will add it to the documentation to make this more clear.

tgrajewski commented 2 years ago

The wiki page still says

NOTE: You do not need a super fast GPU to use these functions. They are useful even on low end hardware, old hardware and embedded OpenGL ES hardware with low performance.

suggesting that mediump would be supported, which is not true.

stegu commented 2 years ago

I don't agree that this suggests mediump support, but I will clarify that passage.

The functions worked on my then-current Samsung Note 1 smartphone which had a GPU inferior to anything you would find even in budget smartphones today, and it ran fine on my contemporary generation of the Raspberry Pi, also a stone age GPU with modern measures. That was the frame of reference I had when I wrote that passage some time in 2015 or so.

Which platforms are you working on that still don't have highp support? My experience is almost exclusively with desktop/laptop hardware, various Samsung smartphones I have owned myself and my wife's iPhones of various older, company-issued models. (Her employer is not a cheapskate, but their security certification of new hardware takes longer than the generation changes, which is a major nuisance.)

Den fre 25 mars 2022 22:10Tomasz Grajewski @.***> skrev:

The wiki page https://github.com/ashima/webgl-noise/wiki still says

NOTE: You do not need a super fast GPU to use these functions. They are useful even on low end hardware, old hardware and embedded OpenGL ES hardware with low performance.

suggesting that mediump would be supported, which is not true.

— Reply to this email directly, view it on GitHub https://github.com/ashima/webgl-noise/issues/25#issuecomment-1079436356, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFGK2VCGOUTAOWWBKN63CTVBYTTPANCNFSM5DXIAJ2A . You are receiving this because you commented.Message ID: @.***>

tgrajewski commented 2 years ago

Thanks, would be great if mediump support would be stated plainly somewhere for this repository. It's very hard to obtain noise/random functions that work with mediump well, if at all.

I'm working on games and targeting mostly mobile devices. From my experience even if a smartphone supports highp it is usually slower than mediump. Also there is still small percentage of devices not supporting highp in fragment shaders at all, but I don't collect details about this. Because of these issues, all my fragment shaders always use mediump.

stegu commented 2 years ago

Sounds like I have a task, then: create some noise functions that work with mediump ! :)

Do these devices have integer support (WebGL 2)? I suppose integer precision is specified separately from float precision, and then the changes required are minimal, provided that integer operations have reasonable performance. (The first desktop GPUs with integer support were dead slow with integer math, so I am not taking this for granted.)

I could probably restrict the code to work with only 16-bit half-floats and no integers, although it will probably require some workarounds that might be slow. I have no clear plan on how to do this, but I'm pretty sure it can be done.

And, not least importantly, how would you suggest I test this? My current Samsung S21 seems to use "highp" no matter what I specify for the precision, but I might be doing my testing wrong. My experience with mobile devices is limited, and I have access only to two smartphones: my own Samsung S21 and my wife's somewhat dated iPhone. Could I perhaps persuade you to provide some assistance in testing? I would be happy to add you as a developer to a Github repo for this purpose.

tgrajewski commented 2 years ago

Hey, awesome! Yes, I'm in :)

Here is a list of some devices that do not support highp at all in fragment shaders:

Unfortunately I don't own any of them, but I have some devices with mediump support, so I can test on those if needed. I'm using WebGL 1, but don't have data on int precision for now.

stegu commented 2 years ago

I got "mediump" working just now on my Samsung S21 (classic stupid mistake: I just needed to wait for the server refresh of the HTML test file I edited), so I will get back to you about testing once I get something working on this one device. So far, I have only broken the function by putting "precision mediump float" at the top, but now I can at least see for myself when (if) I fix it.

Let me just repeat my question from before: is WebGL 2 with a mix of mostly floats and a few integers a reasonable target? That should be relatively quick and easy, while a WebGL 1 compatible version with half-floats only would require more thought.

Also, reading the letter of the WebGL 2 spec it says that "mediump float" could actually be the 10-bit kind-of-floats that I can't see how they could handle anything beyond simple color blending, and "highp float" could be the one that maps to 16-bit "half float". Is that a thing, or can I assume that "mediump" means 16-bit floats? On my Samsung S21, "highp" seems to give me 32-bit "proper" floats in fragment shaders, and with very reasonable performance, and "mediump" seems to be 16-bit half-floats.

tgrajewski commented 2 years ago

WebGL 2 is not good, too low adoption, it needs few more years.

Regarding float precision for mediump, then I think it's the 16-bit half/fixed size most common, yes.

stegu commented 2 years ago

OK. The feature gap between WebGL 1 and 2 (GLSL ES 1.00 vs. GLSL ES 3.00) is a rather wide one, while the differences between GLSL ES 3.00 and the upcoming WGSL are mostly syntactic in nature. For the next few years I guess there will be a need to support two versions of GLSL as well as WGSL. Oh, well. It's not as if "the Web" has ever been a well defined and easy platform to target.

To get a more clear picture of where we stand, I made a test page. When/if you have the time and opportunity, you could check this out:

https://stegu.github.io/psrdnoise/test/showprecision.html

There's no hurry with this, absolutely no obligations, and in any case I don't expect you to give me more than a general hunch, but thanks in advance for any help!

To me, the important information would be whether you see any important devices/platforms that report less than 16 bits for "mediump float", and whether the "int" type has support even where WebGL 2 is reported as "not supported", which would imply that the device is WebGL2 ready even though the browser is not.

It would be interesting to know also if any device supports 32-bit integers but not 32-bit floats. I suppose that's not the case, but I'm just guessing. Actual field testing on real devices is infinitely more useful than guesswork and reading formal specs.

stegu commented 2 years ago

A first attempt at mediump-tolerant 2D noise, in the recently published "psrdnoise" style:

https://stegu.github.io/psrdnoise/test/psrdnoise2-mediump.html It seems to work, but using "mediump" precision has some nasty side effects: mediump texture coordinates makes the pattern blocky if you try to evaluate noise too far from the origin, and when you leave the page running for a few minutes the "time" uniform parameter for animation starts to stair-step and make the animation of "alpha" visibly jumpy. There might be more errors and flaws in there, this is a first attempt.

To avoid overflow in the half-float hash, the repeat tile is smaller, 49x98 units, and the pattern might not be as good looking, because the restrictions for what I could use for the hash function was tight. This is not the final word on the hash, but there are now a lot fewer options to try to improve the "random look" of the pattern. On top of that, the hash function needs a ton of intermediate mod() operations to prevent overflow and truncation in the "half-float-integer" math. Therefore, this is not obviously faster than a highp version, but it might be, and it does work in mediump. Unless you find a way to break it, in which case I will gladly try to fix it. That's what "help with testing" means, after all.

For reference, this is the 32-bit highp version with the "psrdnoise" that is oficially published on that github repo:

https://stegu.github.io/psrdnoise/test/psrdnoise2-highp.html

tgrajewski commented 2 years ago

This is what I get for:

samsung_galaxy_a9

huawei_p30_line

image

image

oppo_reno_3_pro

image

stegu commented 2 years ago

Interesting that compared to that older A9, my reasonably fresh Samsung A21 has added lower precisions (16-bit for both lowp and mediump int and float) to the vertex stage.

I have bothered friends and family with checking the test page as well, and they all report that WebGL2 is supported for both IOS and Android, and that mediump 16-bit floats and highp 32-bit floats are both available for fragment shaders. It's only a sample of a half dozen devices, but I will try putting together a WebGL2-only noise with an integer hash as well. Because it won't need all the mod() operations, it might be considerably faster.

I noticed that the hash in 2D mpsrdnoise needs tweaking for the rotation --- there are too many instances of adjacent gradients pointing in more or less the same direction, and that creates ugly streaks in the pattern. Other than that, do you see any problems with the function?

stegu commented 2 years ago

I mean, of course, S21. All these numbers...

tgrajewski commented 2 years ago

Main obstacle for WebGL 2 is older Safari on iOS devices. Yes, new Safari supports WebGL 2, but from my experience many people still use older Safari v10, v11, etc., without WebGL 2 support.

stegu commented 2 years ago

I thought Safari updates were more or less mandatory on MacOS and iOS devices. So much for "the benefits of a unified ecosystem for developers"...

Updated the hash to improve the look of the 2D mediump-compatible noise. Let me know whether you find it good enough for use.

Moving on to the 3D noise now, but that could be a bit more tricky. The 3D grid transforms and their inverses are more sensitive to low precision. 3D simplex noise might not be a good match for mediump. We'll see.

stegu commented 2 years ago

Benchmarking the 2D mediump noise on a few desktop GPUs, where "mediump" has no effect on speed or precision, it's 15-20% slower because of the extra mod() operations. I'm really not the right person to guess how this translates to mobile GPU performance, but unless mediump and highp are the same speed, I would say the mediump version is quite likely to be faster. Any real world data on this would be appreciated. :)

tgrajewski commented 2 years ago

I can do some benchmarks on the devices I have, but give me few days.

stegu commented 2 years ago

No hurry, no obligations, just: thanks a lot!

tgrajewski commented 2 years ago

I've tested the page https://stegu.github.io/psrdnoise/test/psrdnoise2-mediump.html on my phone and below is screenshot how it looks and looks good! Rendering was constant 60 FPS.

mediump_noise

stegu commented 2 years ago

Well, that pattern should not be difficult even on a fairly weak mobile GPU -- it's just one noise value per fragment. If you want to benchmark by frame rate, I should make a considerably more taxing test page with a sum of several noise components to make the frame rate drop below the 60 FPS cap.

(Better do it right away, or I will forget)

There. This computes 10 noise values per pixel and sums them up to display a slowly changing cloud-like fractal pattern:

https://stegu.github.io/psrdnoise/test/noisebench-mediump.html

The shader is a lot more complicated than it needs to be for that particular pattern, it's written only to require a lot of work for the GPU. My Samsung S21 displays that animation at 60 FPS as well, but we are primarily looking for the phones that struggle with this, which would be 3D-capable budget phones or older phone models. Perhaps we need to increase the workload even more to get any useful results?

Benchmarking is cumbersome if we have to measure it by frame rate. Isn't there any profiling tool that can report on how hard the GPU is working? If not, tell me what you would like to see in terms of benchmarks to give you the information you need as a developer. (If you have any ideas, that is. If not, don't sweat it.)

tgrajewski commented 2 years ago

I again get 60 FPS on this updated benchmark on the same phone.

stegu commented 2 years ago

Hmm. Okay, 10 noise values per fragment, on a 600x600 pixels canvas at 60 FPS, that's still "only" 216 million noise values per second. Most modern GPUs are probably not hitting the limit on that test. I won't dig deeper into this for now, but you can make your own copy of the HTML file (note that an included JS file is required as well, but that's it), and increase the number of repeats for the loop and/or the size of the canvas, to see where a certain piece of hardware hits the limit and drops below 60 FPS. Looking at the raw specs, I think a reasonably modern mobile GPU should be able to compute 1 billion noise values per second, probably more. For comparison, the current top performing desktop GPUs can compute 100 billion 2D noise values per second, in 32-bit precision, which is kind of crazy.

Still working on the 3D mediump noise. The 49-element hash I used for 2D is not quite good enough for a nice "random look" in 3D, at least not without some tweaks I haven't found yet. I think I need to let this simmer on the back burner for a while, and see if I get any better ideas.

Have you tried using the noise for anything yet? It could prove a bit tricky to program procedural patterns that look good if all variables are restricted to mediump. Any stories of real world experiences with this, good or bad, would be useful.

tgrajewski commented 2 years ago

I get 25 FPS on my Galaxy A9 when I modify the canvas to be full screen.

I don't yet have use of simplex/perlin noise, but used white noise found somewhere on the Internet for some shaders.

stegu commented 2 years ago

White noise is certainly useful for some purposes, but band limited, smooth looking pseudo-random noise ("Perlin noise") is great for generating patterns and motion that are quite unlike what we usually mean by "noise": Dirt, wear, fade, scratches, specks, clouds, waves, terrain, fire, smoke, flicker, wind, wrinkles, roughness etc. I'm hoping to see more uses of noise in real time content soon. In offline rendering, it has been used extensively for all the purposes mentioned above, and more, for the past 30+ years, and there's a lot of great examples to learn from as GPU hardware gets up to the task of rendering procedural content.