Finish SPIR-V emitter for MojoShader

[flibitijibibo]

Introductory Information:

MojoShader is a project that takes in Direct3D shader bytecode and emits shaders for other graphics APIs, including ARB1, GLSL, and Metal shaders. It is used by FNA for its Effects framework implementation and is used by every single game using FNA, as well as other non-XNA games like Super Meat Boy, Psychonauts, and Killing Floor.

SPIR-V is the Khronos Group's standard IR for graphics/compute shaders, used by Vulkan, OpenCL, and OpenGL (via ARB_gl_spirv).

The Project:

We need to implement a SPIR-V emitter for MojoShader which works with both OpenGL as well as Vulkan. Currently our priority is OpenGL support since we have all the infrastructure (mostly) set up to work with ARB_gl_spirv program objects already. In the long term, this is primarily for FNA's VulkanDevice, though it is applicable to any game currently using MojoShader that properly uses MOJOSHADER_glBestProfile to autodetect usable shader profiles.

Currently, the MojoShader repo FNA works on is here. However, the current active repository for the SPIR-V work is here. The part you care about is in profiles/mojoshader_profile_spirv.c.

For the most part the SPIR-V emitter works, but we are missing many opcodes (check the file for EMIT_SPIRV_OPCODE_UNIMPLEMENTED_FUNC) and we need to be at least up to feature parity with the glsl120 emitter. glsl120 as well as the other emitters in the profiles folder will be helpful as references. A checklist can be found below.

Lastly, the whole file needs at least one final pass, for polish. Style cleanup would be nice but isn't mandatory.

Prerequisites:

MojoShader is written in C99, so you'll need to be reasonably comfortable with that language to finish this project. Experience with graphics programming is also a plus, but you won't be doing too much of that since you're just touching shader content and not stuff like GL callstreams and what have you.

Knowledge of the Direct3D Shader Model 3.0 specification is highly recommended unless you can read through a whole lot of documents on assembly-like languages very thoroughly. You'll mostly want to read up on the register sets (vs, ps) and instruction sets (vs, ps). MojoShader already handles the bytecode parsing but you'll want to know (at a high level, at least) what the emitter functions are referencing.

You'll also want to read up on the SPIR-V specification, for pretty much the same reasons as the Direct3D shader spec.

Example Games:

At a basic level, lots of FNA games use these Effect files which should work with MojoShader's testparse program, so if you want a quick set of test shaders, this is probably a good place to start. Just about every XNA game uses SpriteEffect; it'd probably be easier to write a list of games that don't use it if I could even remember any.

Examples of more complex Effects include those found in FEZ and Reus, with moderately complex shaders found in Skulls of the Shogun, Apotheon, Cryptark, Escape Goat 2, Murder Miners, and Gateways.

With some LD_PRELOAD magic it may be possible to test SPIR-V support with the Linux/macOS versions of Super Meat Boy and Psychonauts, but this has been untested - maybe ask Ryan C. Gordon about this if you want to hack on those games.

How Much Can flibit Help?

I might be able to help with ARB_gl_spirv integration and maybe help clear up questions about the D3D spec, but I'll confess to not having the best knowledge of the SPIR-V spec. By the time you get test shaders working you'll probably already know more about SPIR-V than me.

Budget/Timeline:

Measuring in weekends, I expect this to take about 2 weekends. The first weekend would be spent on scribbling out the remaining opcodes and the second weekend would be spent on testing/polish. I currently have $500 USD allocated for this project.

[angusholder]

I've been going through MojoShader and API documentation and think I can take this on. I've been able to build the project and run through those example shaders you mentioned from FNA, have gotten testparse.c working and added SPIRV disassembly to it, and now am working on argument swizzles to get ADD working for a start

[flibitijibibo]

Doing a weekly update for this even though it's only been 5-6 days - going to try and keep all these on Friday.

I eventually (sorry!) replied to the DMs, feel free to ping me if something else comes up. I can also get in touch with meklu if we need to figure out what the existing source was doing.

[angusholder]

It's going well now, thanks for the help!

I'm working on it now and have just got ADD, DP3, and DP4 working, along with some plumbing they rely on like uniforms, temp regs stored as globals, and some of the source and dest modifiers. Now that I've worked out the plumbing I needed, putting in the most of the other arithmetic instructions ought to be a breeze. I'm sure I'll encounter some head-scratchers when I get onto samplers though!

[flibitijibibo]

Weekly checkup as usual - we've been in contact throughout the week, just need an overall "what done happened since last Friday" to keep track of what's going on. This next week might be a good time to look in Ryan since he's ultimately the one merging this in.

[angusholder]

So I've implemented the majority of the arithmetic instructions, have linked up (hopefully) all the builtin attributes (the equivalents of gl_PointSize, gl_Position, etc..) and passing attribs from vertex shader to pixel shader, I now handle creating samplers, and have implemented a couple of texture sampling instructions (TEXLD & TEXLDB).

[flibitijibibo]

Doing the weekly update, will save Angus some time this week by just pointing to the diff tracker...

https://github.com/flibitijibibo/MojoShader/pull/6

It's now at the point where basic shaders should work but have only been put through the validator and not a real driver. If you have some shaders, an ARB_gl_spirv driver, and a dash of bravery, feel free to give this branch a try and post what you find.

[flibitijibibo]

Forgot to do weekly updates! Switch work had me busy all weekend, but now I can get back into the generic multiplat stuff. Anything new with this?

[angusholder]

Hey Ethan, sorry I haven't had so much time recently, I managed to get everything hooked up compile-time-wise for using my Mojoshader branch with FNA, at runtime it fails to compile the bytecode though, so my next step is to see what SPIRV glslangValidator produces for equivalent GLSL to see if I'm missing something there, as currently SPIRV-val gives no complaints for what I'm generating for SpriteEffect. I should think I'll be able to get back onto it within the next few days.

[flibitijibibo]

Digging this up... any luck with this lately? Thankfully this is the one bounty that isn't urgent, so we're not pressed for time, but wanted to make sure there weren't any walls that I could help look into.

[flibitijibibo]

Opening this back up - let me know if you can get back into this at some point.

[vitorhnn]

I was messing around with this and I managed to get SpriteEffect's pixel shader working using a test program (not FNA) by using testparse to dump the raw SPIR-V and using ARB_gl_spirv and a vertex shader with compatible location specifiers to load it, which was unexpected, because the disassembly doesn't contain any Location decorators for the color output and glslangValidator requires those to even compile. Maybe this is just Nvidia's driver being lenient, I don't know.

Uniforms also should require Location decorators, which leads me to believe the storage class being used should be changed from Uniform to UniformConstant, as the spec doesn't allow applying Location decorators to the Uniform storage class.

Having shader-defined locations for uniforms and samplers allows us to completely toss away the glGetUniformLocation and glGetAttribLocation reflection currently being done on mojoshader_opengl.c as we can just map DXBC's register numbers to a fixed location, but samplers might complicate that a bit.

[flibitijibibo]

Wouldn't surprise me if NVIDIA's SPIR-V compiler was as lenient as the GLSL, though I wonder if they have warnings somewhere. Maybe the Mesa SPIR-V compiler says something interesting?

Luckily the reflection is pretty easy to skip if we know the ctx is SPIR-V, we can write impl_SPIRV_GetSamplerLocation or what have you to bypass all that.

[infapi00]

Wouldn't surprise me if NVIDIA's SPIR-V compiler was as lenient as the GLSL, though I wonder if they have warnings somewhere. Maybe the Mesa SPIR-V compiler says something interesting?

Hi, I'm Alejandro Piñeiro, one of the developers working on ARB_gl_spirv support for Mesa. Unfourtunately, this is still a WIP. Although we have been pushing to master upstream several patches during the past weeks, there are still work to do. Specifically, UBO/SSBO support needs some finishing and cleaning.

In any case, if you are interested on testing it with Mesa, the last version of the development branch is the following one: https://github.com/Igalia/piglit/tree/igalia/arb_gl_spirv

That one gets the following stats then executing the shaders from the piglit test suite on SPIR-V [34453/34453] skip: 5877, pass: 28353, fail: 210, crash: 13

So although it is still wip, it should cover most of the functionality required (and FWIW, the proprietary NVIDIA drivers gets worse numbers).

FWIW, I subscribed to this issue because eventually we would like to test our driver against a real application, instead of relying on specification tests. So I would appreciate any comment when the progress on this issue is enough to do some testing, Thanks.

[infapi00]

In any case, if you are interested on testing it with Mesa, the last version of the development branch is the following one: https://github.com/Igalia/piglit/tree/igalia/arb_gl_spirv Sorry wrong link, this is the correct one: https://github.com/Igalia/mesa/tree/igalia/arb_gl_spirv

[flibitijibibo]

Mesa is definitely a high-priority target for us as it'll let us target core profile without having to update the GLSL emitters to support specific versions (all we've got is 1.20). I do see gen7+ but just to be sure, is this ready for Haswell too? I do still have my Dev Days kit so I can try this out locally on real hardware as it develops.

[infapi00]

I do see gen7+ but just to be sure, is this ready for Haswell too?

Right now our priority is get it working on gen8+. Having said so, my main development laptop is haswell, and we didn't find any gen7 specific issue yet. So it is really likely that getting it working on gen8+ would get it working (or mostly working) on gen7 too.

[vitorhnn]

I've managed to get SpriteEffect working, though this is a test program that doesn't use MojoShader's Effect framework, since that requires more work on mojoshader_opengl.c to work, particularly uniforms (some tips here would be quite helpful).

I'd also like to apologize for the lack of updates, my midyear break is over and The International is ongoing, so I'm slacking off horribly.

[flibitijibibo]

What does the sample's call stream look like compared to mojoshader_opengl.c's? I was thinking all the work would just be in setup and the existing call stream would be fine, but if we know what the difference is between the two it should help us figure out what changes need to be made. Luckily the Effect stuff won't be affected, just the GL calls.

[krolli]

Hi, I've been playing around with code I found in vitorhnn's repo. Right now I wanted to get all the FNA shaders to generate at least some spir-v without errors to get a grasp on how the spir-v emitter is implemented. I'm almost done with that so next step would be to start getting it working within actual games (or maybe test it with some test programs before that, not yet decided).

I've been wondering, since there have been multiple people working on this, the fork-chain is getting a bit long and I'm not sure whether I should create another repo for my changes or not.

[flibitijibibo]

I wouldn't worry too much about the fork chain - at the end the pull request is going straight to mainline, the chain just lets us know what the whole history is.

[krolli]

Great, so as long as there are no forks I'm not aware of, i'll go ahead and continue on from there.

[flibitijibibo]

krolli's working on this slowly but surely! Marking as In Progress.

[krolli]

Since there has been some reasonable breakthrough, I thought I'd post some update.

I can currently convert all of the .fxb files in FNA without any SPIR-V validation complaints. I have SpriteEffect working in the simplest application (just drawing some sprites) and I have moved on to getting an actual game running on this. OpenGL integration is now in place as well. Uniform management is same as in GLSL back-end (uniforms packed into large arrays) but I have noticed some uniform location issues in shaders. For now, I managed to work around them, but I expect more complicated shaders to run into this again, requiring real fix.

Game I'm testing right now is Bastion since I already have it and I figured it would be fairly simple in terms of shaders being used. Menus are working fine, but once I enter actual gameplay, it gets a bit worse. Things seems to be rendered at much lower resolution and frame time also goes up considerably compared to GLSL (GLSL 16.6ms likely vsync limited, SPIR-V 59ms). From frame debugging GLSL version, I suspect low resolution problem to be somehow tied to bloom or one of the other post-processes.

Unfortunately, attempting to debug or profile SPIR-V version is a bit of a problem right now. Most recent changes that got Bastion working have also caused RenderDoc to crash whenever I attempt to open a capture. Based on minidump, it seems to be crash in whatever is being used to disassemble SPIR-V. I also tried NSIGHT, but it refused to do anything due to calls to glSpecializeShaderARB (required to use SPIR-V modules). I'll probably just grab RenderDoc source code and see if I can fix that crash or, if that doesn't work, work around it in mojoshader SPIR-V generator.

[flibitijibibo]

The BloomComponent (used in a bunch of games including Bastion) does half-size rendering with a GaussianBlur effect, so I wonder if the issue is the sampler parameters - maybe it’s using nearest-neighbor for some reason?

For RenderDoc, if it’s not already set: FNA_OPENGL_FORCE_CORE_PROFILE=1 might fix any capture issues.

[krolli]

The BloomComponent (used in a bunch of games including Bastion) does half-size rendering with a GaussianBlur effect, so I wonder if the issue is the sampler parameters - maybe it’s using nearest-neighbor for some reason?

I don't think sampler parameters are at fault here. I didn't have to do anything different from GLSL for samplers to work in other shaders (though maybe there are some differences I didn't notice). Also, there doesn't seem to be any blur beyond that introduced when you stretch smaller image on larger one. I am more inclined to think that blur passes are broken in some way, leaving image sharp, and that when blurred version is drawn on top of original, alpha gets set to 1 and overwrites the full res version. But I'm only guessing and I'd rather have working debugger, especially since there might be pretty rough problems once I get to 3D effects.

For RenderDoc, if it’s not already set: FNA_OPENGL_FORCE_CORE_PROFILE=1 might fix any capture issues.

Thanks, didn't know about that. It fixed the warning about unsupported compatibility profile, but sadly didn't help with crash when opening the capture. It worked before my last batch of changes that introduced vpFlip uniform and fixed some missing attribute decorations so I suspect something there is causing problems. However, spirv-dis has no trouble disassembling any of my generated shaders, so if RenderDoc is using some older version of SPIRV-Tools for its disassembler, linking against newer version might be enough to fix the problem.

[flibitijibibo]

Not sure if it's relevant but I was looking over some Proton stuff and ran into some issues filed in the DXVK repo related to NVIDIA's newest compiler and SPIR-V performance taking a nosedive - does setting __GL_NextGenCompiler=0 improve performance at all?

[krolli]

does setting __GL_NextGenCompiler=0 improve performance at all?

Doesn't seem to have any effect. Tried with both 1 and 0 and perf. seems to be the same.

I also looked into RenderDoc sources. Unfortunately, crashing code is RenderDoc stuff (not 3rd party lib) and it doesn't seem to be a simple fix. It looks like it doesn't support shaders using UniformConstant and assumes everything is either Uniform Block or Sampler and adding support for that will probably require some integration into UI as well. Maybe @baldurk would be able to shed some more light on this. It's possible I misunderstood things.

I haven't yet looked at what would be needed to use Uniform instead of UniformConstant in MojoShader, but it seems like a possible way to solve this. It wouldn't surprise me if drivers weren't used to seeing UniformConstant either, so they might be generating worse code for it.

[baldurk]

Yes the SPIR-V support is not using SPIRV-Tools at all, and it was written only for Vulkan so it assumes reasonably modern shaders (constant buffers, etc). There may be gaps for GL-only constructs like standalone uniforms which I can fix with a repro case, hopefully it shouldn't be too bad.

When I added SPIR-V support for GL it was a blind implementation, I didn't know of any non-toy applications using it, so the test I set up also used non-legacy shaders.

[krolli]

I could provide a bunch of SPIRV binaries (with GLSL and D3D counterparts) if you'd be interested in expanding support for GL+SPIRV. MojoShader SPIRV backend is still a work in progress and there might be some bugs in terms of functionality, but I try to keep all of them in shape that at least passes validation.

[flibitijibibo]

As a starting point, here are the built-in effects for FNA:

https://github.com/FNA-XNA/FNA/tree/master/src/Graphics/Effect/StockEffects https://github.com/FNA-XNA/FNA/tree/master/src/Graphics/Effect/YUVToRGBA

The SPIR-V branch should be able to parse SpriteEffect at least, haven't checked out the others yet.

[krolli]

It should be able to go through all of them. Here is a zip with generated shaders for different back-ends (makes it easier to see what is going on and helps with spotting bugs) data.zip

fxb - compiled effect files (also has raw SPIRV modules mixed in) d3d - disassembled effect files glsl - output of MojoShader GLSL back-end spirv - output of MojoShader SPIRV back-end spvasm - disassembled raw SPIRV modules from fxb folder

[krolli]

I think I've found out where the problem was. It seems obvious in hindsight, but when assigning locations to uniforms, I only took into account current shader and didn't account for locations being shared between shader stages as well.

Now I just need to rewrite the thing so locations are actually assigned when program is linked. Good news is, this seems to have been the cause of both visual and perf problems.

[baldurk]

I pushed a fix which should let these shaders work. I fixed a couple of GL-isms: textures without explicit bindings, and bare uniforms not in buffers.

If you run into any more cases, drop me the SPIR-V and I should be able to fix it.

[krolli]

Thanks, @baldurk. Just tested it and it seems to be working pretty well. Will definitely let you know if I run into more issues.

[flibitijibibo]

Checking the pulse on this one... at this point it's down to getting all the instructions, right? We should probably get a checklist written up so we know what's needed and what's already done.

[krolli]

There is one final hurdle before it will be down to that, which is a bit tricky and I haven't had much time to dive into it.

Uniform locations in spir-v need to be unique in final linked program and have to take into account size of arrays and such. Basically I need to rewrite location assignment to be postponed until linking for either pixel or vertex shaders. I haven't seen this in any of the other mojoshader back-ends but I think I know how to do it. Sadly, I've been rather busy with other things lately.

After that, I don't know of any other issue apart from missing instructions, but we'll see what we run into when throwing more complicated games at it.

[krolli]

Update: Finally found time to finish the link-time location assignment. Everything still seems to be working fine. I will see what other games using FNA or mojoshader I have and maybe test those to see if I run into some other issues that might complicate things. Beyond that, there seem to be around 40 missing op-codes left to implement: BEM, BREAK, BREAKC, BREAKP, CALL, CALLNZ, CND, DST, DSX, DSY, ELSE, ENDIF, ENDLOOP, ENDREP, EXPP, IF, IFC, LABEL, LIT, LOGP, LOOP, REP, RESERVED, RET, SETP, TEXBEM, TEXBEML, TEXCRD, TEXDEPTH, TEXDP3, TEXDP3TEX, TEXLDD, TEXLDL, TEXM3X2DEPTH, TEXM3X2PAD, TEXM3X2TEX, TEXM3X3, TEXM3X3PAD, TEXM3X3SPEC, TEXM3X3TEX, TEXM3X3VSPEC, TEXREG2AR, TEXREG2GB, TEXREG2RGB.

[flibitijibibo]

I believe a lot of those are tested by this...

https://github.com/FNA-XNA/MojoShader/tree/fna/tests

... but I had to comment this block to get the test to work :|

https://github.com/FNA-XNA/MojoShader/blob/fna/mojoshader.c#L9070

And some of them can be pretty lazy, like EXPP/LOGP:

https://github.com/FNA-XNA/MojoShader/blob/fna/mojoshader.c#L3810

Also, good news! If you're absolutely sick of working on a giant file, we're in the process of fixing that and migrating the SPIR-V stuff to this layout should be nice and simple:

https://github.com/FNA-XNA/MojoShader/pull/5

[krolli]

Size of the file is not that much of a problem (I have survived bigger beasts :P ), though I am thinking of ditching or redoing some macros that generate function signatures. They tend to mess up jump-to-definition in simpler editors and I'm usually not a fan of hiding code in macros this way.

[flibitijibibo]

We’re fighting the same issue - the only thing I’m worried about is accidentally bloating the binary size (which our current patch totally does). I imagine performance issues aren’t going to show up so size will be our focus for optimization.

[TheSpydog]

Hey @krolli I've been looking over your branch and it's really impressive how much you've gotten done! A couple questions/suggestions:

1) Is there anything that could be done to reduce (or remove) the spirv struct inside Context? In the separate-profile-files PR branch above, the context struct is defined in a header and shared with all profile emitters, so I'd prefer to keep it as lean as possible. Maybe we could split it out into a static, standalone spvContext struct or something that could just live inside a mojoshader_profile_spirv.c file?

2) I've noticed there are a lot more spv_* helper functions than for any other emitter. I wonder if we could replace some of them with more generic helper methods, or maybe even remove the need for some of them? Especially the simple ones like spv_emit_word, since essentially all it does is call buffer_append.

Anyways, keep up the good work! 🙂

[krolli]

Hi @TheSpydog , thanks, but a lot of the work was done by people before me. As for your questions:

1. There are a few things that can be done about it. The definition of it certainly doesn't need to be inside Context. I believe noone expected it to get that big initially. All of the usual ways of decoupling those two are doable, but as I was implementing branches and loops, I realized I sometimes used existing members of Context and sometimes needed to expand them with extra information (branch_labels_stack and branch_labels_patch_stack are examples of that). I am wondering whether it might make sense to have a union / opaque pointer to emitter data in Context, but I haven't looked into other emitters to see how useful it would be for them.
2. It is definitely possible and I have some idea how to rewrite type id tracking to make it require less code and be easier to work with. I want to get my current testing set working without any asserts/validation failures firing first though. spv_emit_word along with spv_emit, spv_emit_part and spv_emit_part_va will likely stay though. The most likely candidates for pruning are spv_get{type} functions, just need to redo the whole type id tracking.

I assume this is related to your work on splitting emitters into separate files. I was actually waiting for that change to land in master and then do the separation on spirv branch myself, but I noticed the change is in different repo. Not sure who will be merging between those two.

[flibitijibibo]

The big refactor is in:

https://hg.icculus.org/icculus/mojoshader/ https://github.com/FNA-XNA/MojoShader

Hopefully the isolation of the SPIR-V emitter will make this a little less scary to look at since it's not part of a 10kLOC file >_>

[krolli]

Comparing https://github.com/flibitijibibo/MojoShader and https://github.com/FNA-XNA/MojoShader, I am starting to think I should switch spirv stuff over to one of the other repos. Personally, I am more used to mercurial than git, but I will switch to whichever is more convenient for you to deal with.

[flibitijibibo]

Mercurial would actually be better, since we're ultimately going to push this to upstream before pulling it into FNA's repository.

[krolli]

I moved the changes to mercurial repo. You can find it over at bitbucket (https://bitbucket.org/krolli/mojoshader). I did my best to preserve all previous commits, though some changes (CMakeLists.txt files that were vastly different) have been lost. As a result, there are a few commits that look empty as they only made changes to those files.

Now, on to making it work again in new repo and then merging all the changes from master in last year or so. :)

[TheSpydog]

So what are the next steps with this? Just filling in the missing opcodes and testing? I’ve noticed that some of the other emitters don’t emit anything for obscure opcodes so maybe it’d be good to get a list of the ones we actually need to finish this up. I’ll have some time after this week to help finish this up if you’d like.

[TheSpydog]

I looked at the current source, and it appears we still need to fill in ~26~ ~13~ ~9~ ~8~ ~7~ ~5~ 4 opcodes to achieve parity with the GLSL emitter:

LAST UPDATED: August 28, 2019, 14:00 EDT

• [ ] TEXBEML
• [ ] TEXM3X3TEX
• [ ] TEXM3X3SPEC
• [ ] TEXM3X3

Beyond this list, there are 9 other unimplemented opcodes, but since the GLSL emitter doesn't implement them either... 🤷‍♂

[flibitijibibo]

After looking over the changelog it only occurred to me (read: @TheSpydog reminded me) how many people have worked on this over time, and since we're looking to be close to the "end" (still need to review the working changes but at this point we're happy to just have it work, cleanup can happen during review), I'd like to start distributing the bounty (which now has a larger budget btw) so we can simplify the remaining work for anyone that wants to hammer out the rest of the busy work. @meklu, @angusholder, and @krolli, send a quick e-mail to the address at the bottom of this page so I can start sending these payments out. I can then amend the OP as needed.

[krolli]

Hi, sorry for silence last couple of weeks.

@TheSpydog I've been implementing instructions only when they appear in shaders I test. In other words, I didn't implement anything I can't immediately run through validator. I have a separate repo (https://bitbucket.org/krolli/mojoshader-spirv/src/default/) where I keep stuff for testing. If you'd like to use that, let me know.

Anyway, I'm currently trying to get https://github.com/FNA-XNA/MojoShader/blob/fna/tests/1.vsh working, for which I currently get following errors (assuming check for uninitialized registers is disabled as mentioned before):

ERROR: BREAKP unimplemented in spirv profile
ERROR: CALL unimplemented in spirv profile
ERROR: CALLNZ unimplemented in spirv profile
ERROR: LABEL unimplemented in spirv profile
ERROR: RET unimplemented in spirv profile
ERROR: SETP unimplemented in spirv profile
ERROR: register type p0 is unimplemented
ERROR: unsupported source modifier 13

I plan to have a look at those errors next, which will probably involve doing some infrastructure for function calls.

I suspect that after opcodes, we will still run into some issues with various missing things, but hopefully nothing complicated.