search

pygfx / shadertoy

Shadertoy implementation based on wgpu-py
BSD 2-Clause "Simplified" License
11 stars 1 forks source link

GLSL keywords used as identifiers #22

Open Vipitis opened 7 months ago

Vipitis commented 7 months ago

I am working on the top issues in #15 and InvalidToken shows up quite a bit. There is different variants. But the few I tried to understand were caused by GLSL keywords being used as identifier names. Keywords in Chapter 3.6

On Shadertoy it seems like all identifiers (variable names, function names, etc) get replaced by placeholder variables in the form _name1234 which will avoid this. But the shadercode might make use of certain keywords just fine. The shortest example I found is this shadertoy making use if the buffer keyword as an identifier.

So we very likely need to do some similar sanitizing/obfuscating to support all working shaderstoys from the website as is.

All solutions I can currently come up with, require some sort of lexer or parser to get all the identifiers in the first place, then check them against a mapping of keywords and replace. There will also be further issues if such an approach isn't extremely solid.

On a related note, some examples of InvdalidToken seem to be caused by keywords we introduce with the various compatibility code at the top. So it's probably a good idea to change all names to something more unique than time, mouse or frame.

Korijn commented 7 months ago

What does the website do? Maybe they've developed a very robust regex?

Vipitis commented 7 months ago

There is no proper source, but looking at the scripts inside the website, it looks like the built in code editor has a parser attached that handles keywords. L16392 when you look at the source of https://www.shadertoy.com/new it gets translated to HLSL, you can click on the "Compiled in 0.0 secs" at the bottom and see it.

Another "official" reference is the sourcecode for their iOS app port. But I couldn't find anything specific to replacing keywords, it seemingly just picks the code from the json and does some specific changes: https://github.com/beautypi/shadertoy-iOS-v2/blob/e42f6e7cdd7d85abf0167631eb0d3a28a16cc350/shadertoy/ShaderPassRenderer.m#L137