search

flibitijibibo / FNA-MGHistory

FNA - Accuracy-focused XNA4 reimplementation for open platforms
http://fna-xna.github.io/
246 stars 37 forks source link

[FNA] [HARD] Replace MGFX with MojoShader# #247

Closed flibitijibibo closed 9 years ago

flibitijibibo commented 10 years ago

Difficulty: This task is considered a HARD task. If MEDIUM tasks separate the men from the boys, HARD tasks separate Dark Souls fans from actual serial killers that wear proper armor for reasons yet to be explained.

Right now we're using MonoGame's MGFX format for shader support in FNA. This is problematic for a number of reasons:

MGFX has its purpose for MonoGame, but for an XNA4 preservation project, it's really important that we support the original game data as much as possible, and that means we need to support the Effect files in their original form.

This is nice in that FNA ports no longer need to rebuild, but it's also nice that we don't have to maintain a content building tool as well... you can build effects with either the XNA tools or Microsoft's standalone Effect compiler, fxc.exe.

Something Ryan wrote a few years back for his Shank Linux/Mac ports was initial support for Effect files within MojoShader itself. While it's not complete, it's definitely a good base for what we need to complete this task. I've been working on this in a sandbox over here:

https://github.com/flibitijibibo/EffectRE

I'm basically just taking XNB shaders from the Windows versions of my ports, and taking them apart with UnXNB and MEPTool to ensure that we support the compiled Effect. This is a good place to start if you want to work on getting the file parsing complete.

The next step is to write an API to work with the parsed Effects. The actual shader support should be pretty simple, since that's what MojoShader has always done, but we need to write the entry points to nicely interop with the Effect data in a way that matches the original Effect Framework API. Again, all of this is in MojoShader itself... FNA's not involved quite yet. We're basically finishing Ryan's work and making a solid API for other projects to use, whether they're XNA-based or not.

Where FNA will get involved is when we take out MGFX and replace it with MojoShader interop, using MojoShader#:

https://github.com/flibitijibibo/MojoShader-CS

This is based on the current header file, but as we finish the Effect API, we'll need to update MojoShader# accordingly. If we do our job correctly, most of the FNA-side of things will simply be hooking up the C# API to MojoShader entry points, with minimal fuss overall. Then the GL-specific stuff will simply go into the OpenGLDevice, like all the other GL calls in FNA.

So, here's the task list:

I also need to get Ryan to merge another patch from some work I did a couple years ago ( http://flibitijibibo.com/mojoshader_diff.txt ), but I'll deal with that separately.

CC: @rcgordon, in case Ryan and FNA contributors need to sync up on anything here.

flibitijibibo commented 10 years ago

Some additional work on this is being done separately by @knockoutgames, at this repository:

https://github.com/knockoutgames/mojoshader

Just generic Effect support, nothing XNA-related. That'd be our job anyway.

flibitijibibo commented 10 years ago

For those concerned about replacing the Effect compiler without XNA4, I looked into fxc.exe today.

With some work, fxc.exe works in Wine, too! All you need is this patch...

http://flibitijibibo.com/fna/wined3d_diff.txt

... and with the right prerequisites, fxc.exe should produce Effect files just fine. MEPTool should be able to read them now, too:

https://github.com/flibitijibibo/EffectRE/commit/1e6584070c5f24231bce09fccfb9a85c019c3a5d

So, the official Effect compiler will be able to replace 2MGFX, and it should be able to produce shaders on Windows, Mac, and Linux.

If you're interested in getting this working with upstream Wine, the bug is here:

https://bugs.winehq.org/show_bug.cgi?id=33476

flibitijibibo commented 10 years ago

I've got my own repo for MojoShader changes now, in addition to EffectRE:

https://github.com/flibitijibibo/MojoShader

No GL stuff yet, just trying to get all the data needed by the XNA4 spec, then we can start actually rendering what we parse.

flibitijibibo commented 9 years ago

I just got 100% of the official XNA4 stock effects parsing in my MojoShader branch. Here's BasicEffect parsed out, as an example:

http://flibitijibibo.com/fna/basiceffect.txt

I'm still working out the actual effect API, but once that's settled I'll hook it up to the XNA4 API and get the FNA half rolling.

flibitijibibo commented 9 years ago

Here's the first screenshot of something being run with the XNA4 SpriteEffect, as compiled by fxc.exe and using what should be the official Effect API for MojoShader:

http://www.flibitijibibo.com/images/whatswrongwithyourblobs.png

You can try out the test app yourself if you want, though you may have to fight the CMake file on non-Linux platforms:

https://github.com/flibitijibibo/EffectRE/tree/master/spritetest

There are a couple things I want to do before getting the FNA side done:

https://github.com/flibitijibibo/EffectRE/blob/master/FNA/StockEffects/HLSL/BasicEffect.fx#L585 http://www.flibitijibibo.com/fna/basiceffect.txt (see the bottom two blocks)

So I need to figure out how that instruction list is supposed to result in array selection, which is all this really is... grumble. EDIT: Now completed! basiceffect.txt now notes the array accesses, too.

https://github.com/flibitijibibo/MojoShader/blob/master/mojoshader_opengl.c#L2673 https://github.com/flibitijibibo/MojoShader/blob/master/mojoshader_opengl.c#L2743

But the rest is actually pretty decent. Wasn't expecting that.

There are one or two FIXMEs in the parsing as well, but they're pretty obscure... anyone ever used an entire struct as an EffectParameter?

https://github.com/flibitijibibo/MojoShader/blob/master/mojoshader_effects.c#L338

flibitijibibo commented 9 years ago

Managed to figure out what the preshader array accesses were. It turns out, on top of the source register, there's extra data where each uint32 refers to a base register index of the target arrays (which can be nested, of course), and within the uint32, the first two bits are ignored, the next two bits refer to register_index % 3, and the rest of the uint32 refers to register_index / 4. Because of course it would:

https://github.com/flibitijibibo/MojoShader/blob/master/mojoshader.c#L9839

All that's left is the array access work in the VM and this is ready to be tested in FNA:

https://github.com/flibitijibibo/MojoShader/blob/master/mojoshader_effects.c#L76

Optimizations are definitely still something to do, particularly in CommitChanges... BeginPass has one possible optimization, but CommitChanges is pretty insane at the moment.

flibitijibibo commented 9 years ago

I now have every feature I wanted to get out of MojoShader working. The rest of this week I'll be working to hook it up to FNA in a new branch; once I've got it working I'll bump this with a pull request showing the changes and giving developers a chance to test with the new shader backend.

This is what it took in MojoShader, apparently (in addition to the glVertexAttribDivisor patch):

https://github.com/flibitijibibo/MojoShader/compare/bd6b1a7...HEAD

flibitijibibo commented 9 years ago

Initial work on the Effect rewrite has started:

https://github.com/flibitijibibo/FNA/tree/mojoshader-cs

At this point, SpriteBatch with/without custom shaders should work. It's not very optimized, but if you have noticeable performance issues, throw some numbers at me.

Some notes for myself in case I forget...

Port Checklist...

flibitijibibo commented 9 years ago

mojoshader-cs now deals with almost all of the render states, and all known compatible sampler states. At this point, most of my ports now work with the original shaders, using no modifications for FNA in particular. If your game uses SpriteBatch for rendering, this should basically work now; fully custom renderers might run into problems similar to what you see in my TODO list above. Right now performance is going to be pretty laughable because my state application generates tons and tons of new states when it probably doesn't need to.

Something I should note at this point is that we are not doing any of the half-pixel offset hackery previously done by the MGFX shaders. This actually fixed a lot of visual issues, but in exchange, there's a chance that some games that account for the half-pixel offset will actually break. Take, for example, @kg's Squared.Render bitmap shaders:

https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L9 https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L105 https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L128

While this previously worked with the old system, the new system now introduces a noticeable pixel offset in EG2, for example. It's fixed pretty easily (just remove the 0.5 subtraction), but it's something that's worth noting. I'm on the fence about trying to account for this in FNA, because it does avoid the obvious visual error, but the tradeoff is making absolutely everything else just off enough to make a nearly unfixable difference for games that care about per-pixel accuracy, which we can't ever fix by having this sort of workaround. In the case of client-side bugs like EG2's, it's at least fixable.

We even do this ourselves in FNA; The XNA4 SpriteBatch adds the pixel offset to the TransformMatrix parameter, but we do not. This is less a problem with XNA accuracy and more a problem with how various rendering backends might care about this offset; it's probably worth making a GLDevice.HalfPixelOffset value for each driver to return, but for now, I'm just saying screw it and leaving the offsets out.

kg commented 9 years ago

Is there a way to get a preprocessor def so that we can do '#ifdef FNA' in our shaders to patch out half pixel offsets and such?

On 23 November 2014 at 19:05, Ethan Lee notifications@github.com wrote:

mojoshader-cs now deals with almost all of the render states, and all known compatible sampler states. At this point, most of my ports now work with the original shaders, using no modifications for FNA in particular. If your game uses SpriteBatch for rendering, this should basically work now; fully custom renderers might run into problems similar to what you see in my TODO list above. Right now performance is going to be pretty laughable because my state application generates tons and tons of new states when it probably doesn't need to.

Something I should note at this point is that we are not doing any of the half-pixel offset hackery previously done by the MGFX shaders. This actually fixed a lot of visual issues, but in exchange, there's a chance that some games that account for the half-pixel offset will actually break. Take, for example, @kg https://github.com/kg's Squared.Render bitmap shaders:

https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L9

https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L105

https://github.com/sq/Fracture/blob/master/Squared/RenderLib/Content/BitmapCommon.fxh#L128

While this previously worked with the old system, the new system now introduces a noticeable pixel offset in EG2, for example. It's fixed pretty easily (just remove the 0.5 subtraction), but it's something that's worth noting. I'm on the fence about trying to account for this in FNA, because it does avoid the obvious visual error, but the tradeoff is making absolutely everything else just off enough to make a nearly unfixable difference for games that care about per-pixel accuracy, which we can't ever fix by having this sort of workaround. In the case of client-side bugs like EG2's, it's at least fixable.

We even do this ourselves in FNA; The XNA4 SpriteBatch adds the pixel offset to the TransformMatrix parameter, but we do not. This is less a problem with XNA accuracy and more a problem with how various rendering backends might care about this offset; it's probably worth making a GLDevice.HalfPixelOffset value for each driver to return, but for now, I'm just saying screw it and leaving the offsets out.

— Reply to this email directly or view it on GitHub https://github.com/flibitijibibo/FNA/issues/247#issuecomment-64147983.

flibitijibibo commented 9 years ago

I know fxc.exe lets you do something like -DFNA, but I forget the syntax. Not sure about the XNA4 pipeline though.

flibitijibibo commented 9 years ago

I just got my last port working with the Effect rewrite - at this point, it should just be a matter of cleaning things up and optimizing the more painful functions. But, you can now expect this to at least work out of the box, unless there is a case I've missed.

Because of the sheer size of this rewrite, I will not be merging this in until my last port for the year ships. That port is going to be my benchmark for this rewrite, because it's kind of a giant monolith and will take a bat to this work better than any of my previous work ever will. I still invite devs to give this branch a try if they're willing to self-build the MojoShader branch, which is still over here:

https://github.com/flibitijibibo/MojoShader

flibitijibibo commented 9 years ago

I have opened up the pull request for the MojoShader# Effect rewrite:

https://github.com/flibitijibibo/FNA/pull/269

I start full-time work on my last port for 2014 tomorrow, so focus on this component will slow down a bit unless the port requires more work here, be it in features or optimization. I expect to merge this in when I have released the port.

Because I now have all of my games working with this, most of the remaining features are for things I don't have a reference for, or are things that I'd like to optimize once I have this last game working correctly. So, here is a summary of all the FIXMEs remaining in the rewrite (you can grep around for "FIXME" and "-flibit" to find them in the actual patch):

Lots of tiny FIXMEs in Effect.INTERNAL_applyEffect(). Just go to that method and you'll see what I mean. Each one is pretty detailed, so I won't duplicate it all here. In general, though, the performance here kind of blows, but it's in exchange for improved accuracy in state changes (something MGFX struggled with, at least for my games). Things like state caching are something we really need here, otherwise performance noticeably degrades over time. Sadly I expect this rewrite to be slightly slower than MGFX overall, but only because we do more work to be more accurate here. That doesn't mean it should be slower for dumb/stupid reasons, however.

EffectParameter.StructureMembers will always be null. This is a problem with my MojoShader work, not really FNA. Go look at readvalue in mojoshader_effect.c and you'll find the empty block where struct parsing should be. I don't have any game that uses entire structs as parameters, so I didn't have any reference blobs to parse.

Effect Collections return null when they cannot find a string in the Collection. For example, if you call myEffect.Parameters["NotExistant"].SetValue(0); and "NotExistant" is not found, we just return null and a NullReferenceException is thrown. I would think that we'd throw an exception rather than silently fail, but the MSDN documentation gives us no indication on what might be thrown. If anyone knows the XNA4 behavior for this, let me know.

GetValueMatrix and SetValue(Matrix) assume 4x4 matrices. Basically we just have to go through the arduous process of handling all of the combinations of matrix sizes (min size is 1, max size is 4). I've only ever seen 4x4 though, so that's just what I implemented.

GetValueString and SetValue(string) are not implemented. GetValueString requires that we make a copy of the string from the native MOJOSHADER_effect's object list. This probably isn't hard, but then there's SetValue(string)... who would even use this in their shaders? If there's a reason someone might do this and it actually affects the shaders somehow, I'd like to know so I can implement this properly...

GetValueQuaternion and SetValue(Quaternion) are not implemented. This isn't because there's a missing feature, it's because I dunno how these are stored in the constant buffer. Is this even an HLSL type?

Vertex attributes/divisors are not tracked, so we will always call glVertexAttribPointer/glVertexAttribDivisor for each draw call. Previously, because we handled the attributes directly, we could be smart about caching attributes given buffer bindings and the current shader program. But, because that's hidden behind MojoShader now, we can't be quite as smart about this anymore. The tradeoff is that the GLDevice API looks SO much better now, but it'd be nice to get the number of GL calls back down again.

VideoPlayer still uses direct GL calls and raw GLSL, rather than the GraphicsDevice and D3D shaders. There are still some enums/functions in the OpenGLDevice that we store solely for the VideoPlayer, because I never got this using the GraphicsDevice for draw calls, textures/buffers, etc. I would much rather do that for both portability and stability's sake. The only part where it gets fuzzy is the YUV->RGBA shader... it'd be a very simple Effect, but a part of me wishes I could just use a damn D3D shader and restore the shader program state after the video texture's been made. The MojoShader Effect API can actually do this for us, but I dunno if it's worth the extra method calls. Performance in the VideoPlayer is a really big problem, and I'm a bit antsy about making it any worse.

Developers that can address any of these problems should try the branch out and see if they can fix them. I could attempt to fix some of these, but without a reference I'm doing it blindly. Additionally, I'm still working on Linux primarily, so Windows may require some work to get the native MojoShader library building, and OSX may require one or two changes (probably all to the CMake file). If you do make changes, send pull requests over at that repository:

https://github.com/flibitijibibo/MojoShader

Additionally, anyone that wants to hack around using one of my ports with the Effect rewrite can ping me, and I'll make a branch on Steam that uses the rewrite. From there, it should be possible to self-build your own FNA lib and use that to work on the game that way.

flibitijibibo commented 9 years ago

I just got the Windows build of MojoShader.dll working:

https://github.com/flibitijibibo/MojoShader/commit/a260b9a39b9e1d799e730f06d4b73560d22ee6a6

Because of complicated process needed to get it building on Windows (read: /TP because the VS C compiler blows), I've provided a precompiled binary here for Windows developers to test with:

http://flibitijibibo.com/MojoShader_Win32.zip

At this point I've got OhGodNo almost working, and if MojoShader# works with that game, it's pretty safe to say yours will as well. Just a few little oddities here and there, and then this will probably slow down while I get everything else working for that port.

flibitijibibo commented 9 years ago

Just updated MojoShader and MojoShader_Win32 for the first time in a while... this marks Apotheon as working with the MojoShader# rewrite as well. Codename OhGodNo is still too messy for me to tell where the real bugs are, but we're definitely getting close to this getting merged. I've decided to merge when either of those two ports release, since I don't know what the order's going to be anymore.

flibitijibibo commented 9 years ago

Since I'm hosting these for Apotheon anyway, here are some Linux/Mac binaries too:

http://www.flibitijibibo.com/mojoshader32.tar.bz2 http://www.flibitijibibo.com/mojoshader64.tar.bz2 http://www.flibitijibibo.com/mojoshaderOSX.tar.bz2

Now it should be easier for everyone to test mojoshader-cs, which should be getting merged in the next few weeks.

flibitijibibo commented 9 years ago

This will be merged into master when Apotheon Linux/Mac ships:

https://plus.google.com/+flibitijibibo/posts/FWyMi7Ex14u

flibitijibibo commented 9 years ago

Apotheon Linux/Mac is now live, so this is now in mainline:

https://github.com/flibitijibibo/FNA/commit/8bc84bfc51436f89a99ce77c62bacb68d236069d

(Don't open this link unless you're nostalgic about Crysis on the machine you owned in 2007)

Aranda commented 9 years ago

I've provided a precompiled binary here for Windows developers to test with: http://flibitijibibo.com/MojoShader_Win32.zip

Have you still got some MojoShader binaries hosted? I'm testing out Square Heroes with the very latest FNA master and it seems I need them. Also wondering if this should Just Work (tm) with existing effects built with 2MGFX?

flibitijibibo commented 9 years ago

With the Effect rewrite now in, the binaries are now in the usual fnalibs archive:

http://www.flibitijibibo.com/fnalibs.tar.bz2

Also, MGFX files are no longer supported.

Aranda commented 9 years ago

Thanks Ethan.

BTW, not sure if this is a bug or expected, but it seems the latest MonoGame content changes don't support legacy Xml assets compiled with Xna's auto-Xnb-serialization. That's my guess anyway as I'm getting an incorrect ContentTypeReader index from Read7BitEncondedInt() in ListReader.Read().

flibitijibibo commented 9 years ago

@Aranda If you can identify the revision where it broke I can check it out. Odds are it's a MonoGame-specific thing, since I've not had to touch the Content namespace in quite a few months. List history, for reference:

https://github.com/flibitijibibo/FNA/commits/master/src/Content/ContentReaders/ListReader.cs

If it breaks and it's XNA4 content, the MonoGame team will want to know.

Aranda commented 9 years ago

Ok scratch that last comment. I think my content file is just out of date :|

flibitijibibo commented 9 years ago

Hehe, it happens.

If it's built with the MonoGame pipeline, you may have to update on their side as well. @tgjones in particular has been doing some mighty fine work both on the pipeline side and on general XNA accuracy things, so if your stuff is built with their tools, it'd be worth trying with XNA4 as well to see if they're on the right track.

Aranda commented 9 years ago

Yeah I've been following the pipeline tool's progress with much interest, but haven't made the switch yet (still living in dummy XNA4 game plus bat-file copy land).

Incidentally, how do I go about compiling effects for your MojoShader change?

flibitijibibo commented 9 years ago

If you're building with XNA already, no change is necessary. Just use those XNB files and you're all set. When outside of the pipeline, you can build with fxc.exe, found in the DirectX SDK.