doitsujin / dxvk

Vulkan-based implementation of D3D8, 9, 10 and 11 for Linux / Wine
zlib License
12.85k stars 830 forks source link

Unigine Superposition launcher crash #70

Closed nairaner closed 6 years ago

nairaner commented 6 years ago

Software information

Unigine Superposition launcher crashes in kernel32

System information

Apitrace file(s)

https://mega.nz/#!5YI2VQAB!p0069ISZdqoeMSXdPX20fJXlQQM8-C36ZjWhtYl48To

Log files

SveSop commented 6 years ago

Possibly the "D3D11Texture2D" interface creation that is not implemented yet causing this? Get the same as you, and doing the spirv-opt trick on the 2 dumped shaders does not help for this problem.

warn:  D3D11Texture2D::QueryInterface: Unknown interface query
warn:  D3D11Texture2D::QueryInterface: Unknown interface query
warn:  D3D11Texture2D::QueryInterface: Unknown interface query
debug: Compiling shader VS_cdfd077b92b7b670e3a0bbf1f30a5e771d521c63
debug: Compiling shader PS_9726ef301ae6484bf30d792aacfe3fd27fa4e83c
warn:  D3D11Texture2D::QueryInterface: Unknown interface query

Not sure tho... "warn:" should possibly not be gamebreaking? :)

doitsujin commented 6 years ago

Will have to take a look at which interfaces it actually needs, but this sounds like a lot of work. In the worst case it may even require something as stupid as DXGI<->GDI interop. It's likely that the crash is related to that though.

SveSop commented 6 years ago

There is currently a bug generally in wine with the SuperPosition gui. Its relatively "garbled", but if you know appx where the buttons are, it works.

However, if you set "d3d11.dll" to "disabled", it will switch to DX9 (i think), and work fine.. but the benchmark will ofc not work for "DirectX". This bug is with ALL versions of Wine, so not really related only to running with dxvk dll's.

So it might be more to it than fixxing D3D11Texture2D transformation perhaps?

Bugzilla: https://bugs.winehq.org/show_bug.cgi?id=43683

oscarbg commented 6 years ago

Hi guys, I got it working more or less on Vega! on native Win10 RX Vega.. the trick is not put d3d11.dll and dxgi.dll and launch the launcher.. of course it launches correctly and then selected custom medium quality windowed 1080p and before running putted back dxvk dlls on the folder (latest as of today) see results: imagen

superposition_d3d11.log superposition_dxgi.log

seems logs complain a lot of: D3D11Device::CreateGeometryShaderWithStreamOutput: Not implemented CreateBuffer: D3D11_BIND_STREAM_OUTPUT not supported D3D11DeviceContext::SOSetTargets: Not implemented finally a warn about: warn: DxbcCompiler: Unhandled opcode class: DxbcOpcode::BfRev

adding launcher logs anyway: launcher_dxgi.log launcher_d3d11.log which also complain about D3D11Device::CreateBuffer: D3D11_BIND_STREAM_OUTPUT not supported

pingubot commented 6 years ago

thx for the tip, after optimizing the shaders it also fully works on nvidia. As far as i can tell, it renders completely correct (plain wine is not able to do that). I am again really impressed whats already possible at this stage of the project.

ZeroFault commented 6 years ago

@doitsujin

I'm getting these validation errors from spirv-val on these spir-v files in the Superposition benchmark.

PS_ae8e6b1471b25a5db1b500dee170dc3a122bcd6c.spv
error: 514: Illegal number of components (5) for TypeVector
PS_b894ff8ee03e2fe3bf870fe31502f9b7d1b50aeb.spv
error: 376: Illegal number of components (5) for TypeVector
PS_d7347a5c4bb3b87ffab4c4eef3e49364215baf07.spv
error: 530: Illegal number of components (5) for TypeVector
PS_f3075e40e8dee2daa325b9c63685f1215fc5da6d.spv
error: 514: Illegal number of components (5) for TypeVector

invalid-spirv.zip

doitsujin commented 6 years ago

@ZeroFault Are you on Nvidia hardware?

Thing with Nvidia is that their driver is broken and requires packing the depth value into the component vector for depth-compare operations, which seems to cause trouble here since cube map array coordinates are already four-dimensional. I don't know if it is possible to fix this issue without breaking Nvidia support entirely.

You can try commenting out the following line in dxbc_options.txt:

      this->packDrefValueIntoCoordinates = true;
SveSop commented 6 years ago

@ZeroFault I found that adding --skip-validation helped for those TypeVector errors. spirv-opt --inline-entry-points-exhaustive --skip-validation --eliminate-dead-code-aggressive

@doitsujin I mentioned washed out colors on the other thread.. superposition

SveSop commented 6 years ago

The attached 3 shaders dumped from Superposition fails validation with spirv-val. To be able to optimize those to work with nVidia, they need --skip-validation

Since im quite the n00b here, is it the driver that dump those .spv files, or is it dxvk?

broken.tar.gz

ZeroFault commented 6 years ago

@doitsujin Yes I am using Nvidia. Unfortunately, disabling the packDrefValueIntoCoordinates causes issues with rendering. I wish Nvidia would stop having so many issues with its driver.

@SveSop DXVK is dumping the .spv files

pingubot commented 6 years ago

@doitsujin : have you opened bug tickets at nvidia already for all the issues which you already found ?

oscarbg commented 6 years ago

HI guys, some updates( all on Win 10 native): 1) implementing bfrev fixed graphics issues on AMD Vega now renders correctly at medium and high settings.. 2) tested also on Nvidia fixing the shaders (spir-opt legalize-hlsl) without the bfrev commit and also exact graphics issues were present but after that patch also renders perfectly on Nvidia..

so basically two issues to say: 1) settings shaders at maximum quailty (maybe called ultra) is only path that renders with graphical issues.. maybe that's the only path making use of geometry shaders and hitting this issues seriously: D3D11Device::CreateGeometryShaderWithStreamOutput: Not implemented CreateBuffer: D3D11_BIND_STREAM_OUTPUT not supported D3D11DeviceContext::SOSetTargets: Not implemented

2) with a bat file I legalize shaders with something like: for /f %%f in ('dir /b *.spv') do spirv-opt %%f --legalize-hlsl -o %%f.leg @ZeroFault @SveSop @doitsujin I'm using the spirv-opt executable included in the new Vulkan SDK 1.0.68 for Windows: and as I say I don't see any errors like: error: 514: Illegal number of components (5) for TypeVector and also don't need to use --skip-validation argument in spirv-opt as I say it runs correctly with: spirv-opt %%f --legalize-hlsl -o %%f.leg

may be spirv-tools may diverged a bit between the included in SDK and in github master.. I get in SDK: spirv-opt --version SPIRV-Tools v2018.1-dev v2018.0-6-g9e19fc0

oscarbg commented 6 years ago

@pingubot @doitsujin don't worry, I, as crazy as I am :-), bombarded all NV Vulkan dev techs I know pointing to this issue: https://twitter.com/oscarbg81/status/964301753184333824 but if it's not enough let's ping also to his github names: @pdaniell-nv @pixeljetstream what do you say about:

Thing with Nvidia is that their driver is broken and requires packing the depth value into the component vector for depth-compare operations, which seems to cause trouble here since cube map array coordinates are already four-dimensional. I don't know if it is possible to fix this issue without breaking Nvidia support entirely.

also that some shaders that are consumed correctly by AMD closed driver or even RADV fail on Nvidia without doing some: spirv-opt --inline-entry-points-exhaustive --skip-validation --eliminate-dead-code-aggressive on this shaders.. even with that or a full "spirv-opt --legalize-hlsl" pass some shaders still present issues..

pdaniell-nv commented 6 years ago

What specific SPIR-V OpImage function is having the issue? Eyeballing our shader compiler it appears dref is picked up as a separate operand, rather than the 4th component of the coord, from functions like OpImageDrefGather. What driver version are you using?

pdaniell-nv commented 6 years ago

Could someone share a crashing SPIR-V shader with me. It wasn’t immediately obvious to me how to get the project going under windows so just a problematic shader is probably the easiest thing to debug at this point. Thanks.

oscarbg commented 6 years ago

@pdaniell-nv IMHO the fastest way to test is this:

a)test on Windows to avoid setting up Wine-staging or from https://github.com/roderickc/wine-vulkan b) if you don't want to compile by yourself download precompiled binaries from: https://haagch.frickel.club/files/dxvk/ for example from latest commit download both d3d11.dll and dxgi.dll from: https://haagch.frickel.club/files/dxvk/r608.8ebffc1/64/bin/ or https://haagch.frickel.club/files/dxvk/r608.8ebffc1/32/bin/ as needed by the app..

if you don't trust this guy, you can also build DXVK from Windows (basically to obtain the 2 DLLs this project is about (d3d11.dll and dxgi.dll) for 64 bit DLLs: I download https://nuwen.net/mingw.html launch his shell with compilers in path and then clone this project and modify build-win64.txt: comment "exe_wrapper = 'wine':"

exe_wrapper = 'wine'

and remove "/usr/bin/x86_64-w64-mingw32-" from all this lines: c = '/usr/bin/x86_64-w64-mingw32-gcc' cpp = '/usr/bin/x86_64-w64-mingw32-g++' ar = '/usr/bin/x86_64-w64-mingw32-ar' strip = '/usr/bin/x86_64-w64-mingw32-strip' so basically leaving c = 'gcc' ,etc.. then following his instructions you should obtain the 2 DLLs ..

c) put the dlls in the executable to test.. altough I pointed to this issue because of developer talking about Nvidia problems I would suggest for a simple failing program test ComputeMark2 program.. you have to trust the file I uploaded because this program has dissappeared from the web altough it was popular by review sites for new GPUs in the early D3D11 days (2010s): https://github.com/doitsujin/dxvk/files/1670646/ComputeMark2.zip

it's a 32bit app so 32bit DLLs..

it a good test because has very few shaders (less than 10) mostly compute shaders that all fail to compile under Windows 10 17083+Nvidia GTX 970 +390.65 drivers.. start a cmd.exe: set DXVK_SHADER_DUMP_PATH=c:\dxvk\shaders set DXVK_SHADER_READ_PATH= set DXVK_HUD=1 (optional but to verify the D3D11 it's using Vulkan) ComputeMark2.exe

that should get spv files and dxbc files c:\dxvk\shaders using spirv-opt.exe from Vulkan SDK 1.0.68 I run on all shaders: (with a script) spirv-opt --legalize-hlsl file.spv -o file.spv.legal and copy spv.legal to .spv

my bat is

for /f %%f in ('dir /b *.spv') do copy /y %%f %%f.ori >nul
for /f %%f in ('dir /b *.spv') do spirv-opt %%f --legalize-hlsl -o %%f.leg
for /f %%f in ('dir /b *.spv') do copy /y %%f.leg %%f >nul

and then run the program but with these legalized shaders: set DXVK_SHADER_DUMP_PATH= set DXVK_SHADER_READ_PATH=c:\dxvk\shaders ComputeMark2

you should continue seeing a windowed black screen and logs that it generates show failing creating shaders o compute pipelines don't remember..

I run spirv-opt --legalize-hlsl because that is shown to fix issues on NV GPUs on Unigine Heaven and Valley benchmarks (i.e. this SPIR-V programs weren't compiling successfully but with this pass they make NV SPIR-V compiler happy).. in practice means going from the app running but seeing a black screen to rendering perfectly fine..

as said ComputeMark2 for example runs fine on same system with DXVK on AMD Vega GPU even without spirv-opt --legalize-hlsl step..

if you wait a day or so I will upload the ComputeMark2 spv shaders with and without HLSL legalization so you can check what's going wrong.. altough I recommend checking by yourself for example that ComputeMark2 tests run and visualize correctly..

hope ComputeMark2 fixing triggers fixing most or all other remaining NV issues modulo bugs in DXVK project of course..

GabrielMajeri commented 6 years ago

@pdaniell-nv You can also get some DXVK binaries from OpenSUSE's OBS.

This archive contains all the vertex and fragment shaders for Unigine Valley (there's a lot of them, I'm not sure which one is problematic). The .spv files are SPIR-V files, the .dxbc files are the Direct3D11 shader files. The game launches but shows a black screen, and DXVK says it failed to create the graphics pipelines.

shaders.zip

oscarbg commented 6 years ago

@GabrielMajeri @pdaniell-nv this is good to test but not perfect because as has said before with spirv-opt --legalize-hlsl this "NV problematic" spirv shaders get fixed.. instead Computemark2 shaders are less than 10 and continue being problematic on Nvidia driver even after "legalized" with spirv-opt --legalize-hlsl... so I believe NV must see why hlsl-legalized spv shaders don't compile fine and then attack the spv shaders that can be fixed by spirv-opt --legalize-hlsl.. of course just my IMHO..

Guy1524 commented 6 years ago

@pdaniell-nv Here is one example of a shader which requires spirv-opt on NVIDIA from Unigine Valley:

W/O spirv-opt (crashing): https://paste.ubuntu.com/p/SqyDxQbKwn/

W/ spirv-opt (not crashing): https://paste.ubuntu.com/p/xDgZXbhfWc/

oscarbg commented 6 years ago

@pdaniell-nv @doitsujin @SveSop @pingubot @Guy1524 @GabrielMajeri @ZeroFault

just found that AMDVLK latest 2.0.15 open source driver on Vega build runs Superposition (previously ones didn't launch correctly) and what a surprise(!) also need "hlsl-legalization" (spirv-opt --legalize-hlsl) on some shaders to render correctly.. so very similar to Nvidia driver and this just proves that we just were lucky with AMD RADV and closed AMDGPU PRO driver in that accepted more relaxed SPIR-V shaders.. still Nvidia has more problems than even AMDVLK seems..

pdaniell-nv commented 6 years ago

In the https://paste.ubuntu.com/p/SqyDxQbKwn/ shader, it's causing a crash because there is "OpTypeImage %float 3D 1 0 0 1 Unknown" in the shader that we don't handle. There is no crash in https://paste.ubuntu.com/p/xDgZXbhfWc/ because the dead-code elimination has removed it since it's not referenced anywhere. We have no support for depth-sampling a 3D image so never expected to see a 3D image created with depth set. Any ideas why this OpTypeImage is being emitted?

ZeroFault commented 6 years ago

Hmm, looks like a bug in the DXBC to SPIRV compiler to me.

pchome commented 6 years ago

https://github.com/doitsujin/dxvk/search?utf8=%E2%9C%93&q=defImageType&type=

oscarbg commented 6 years ago

one more post here today.. I tested forcing disabling depth bounds test and 2 other caps requeriments for DXVK working on Intel HD530 with Anvil Mesa driver and tested lots of games.. good news is compatibility is very high.. lots work OK.. even Superposition and doesn't need hlsl legalization.. just as good as RADV.. proof: igpusuper2

doitsujin commented 6 years ago

@pdaniell-nv DXVK generated those depth types for all sampled image types. As far as I'm aware, it's not illegal to do that in SPIR-V, although certainly unnecessary if the type is not going to be used anyway. In any case, thanks for the information - I restricted that to 2D and Cube textures now though, so this particular problem should be fixed.

There are however more issues. I don't have an Nvidia card myself to test at the moment, but an example of a compute shader that apparently crashes the driver is https://pastebin.com/6cmCE49u this one. Inlining the cs_main function seems to do the trick in this case.

@oscarbg good job. Disabling depth bounds test on Intel is something I had in mind as a potential workaround, but the other two (image store/load without format) are a bit more problematic.

nairaner commented 6 years ago

@pdaniell-nv I already reported this example in Nvidia customer care, but all you need to do is start start.bat file in debugger

https://mega.nz/#!ZJI31JwS!k2grgsQwxmvojVvhqhSnHYIQkv30Yr5FHRqfrbs6LbI

  1. There are 3 different shaders in bin\example\shader_opt:
    • default produced by dxvk (crash): CS_3e57aa98bdc58df0415b95c62877ef9e630eff9f.spv_unopt
    • optimized using SPIRV Tools (no crash): CS_3e57aa98bdc58df0415b95c62877ef9e630eff9f.spv_opt
    • modified by removing inline (no crash): CS_3e57aa98bdc58df0415b95c62877ef9e630eff9f.spv_mod

Disassemblies:

  1. Replace wanted shader with CS_3e57aa98bdc58df0415b95c62877ef9e630eff9f.spv

  2. Start program using bin\example\start.bat

pingubot commented 6 years ago

@doitsujin : just tested heaven and superposition with the latest dxvk build and it still crashes here on nvidia.. Should we move the discussion her to thread #64 ?

Edit: trying to get heaven to run again, i noticed that i needed much less rounds of optimizing the shaders to get it running, so your code change indeed fixed some of the crashing issues it seems.

Edit 2: Same applies for valley. I needed to optimze the shaders only one time, the rest works fine. In the past it crashed a dozen times+ a dozen round optimizing was necessary. So it looks like only some minor issue may be left now.

SveSop commented 6 years ago

@oscarbg

with a bat file I legalize shaders with something like: for /f %%f in ('dir /b *.spv') do spirv-opt %%f --legalize-hlsl -o %%f.leg @ZeroFault @SveSop @doitsujin I'm using the spirv-opt executable included in the new Vulkan SDK 1.0.68 for Windows: and as I say I don't see any errors like: error: 514: Illegal number of components (5) for TypeVector and also don't need to use --skip-validation argument in spirv-opt as I say it runs correctly with: spirv-opt %%f --legalize-hlsl -o %%f.leg

If you look at spirv-opt --help, you can see that --legalize-hlsl does the following:


--legalize-hlsl
Runs a series of optimizations that attempts to take SPIR-V
generated by and HLSL front-end and generate legal Vulkan SPIR-V.
The optimizations are:
    inline-entry-points-exhaustive
    eliminate-dead-functions
    private-to-local
    scalar-replacement
    eliminate-local-single-block
    eliminate-local-single-store
    eliminate-local-multi-store
    simplify-instructions
    eliminate-dead-branches
    eliminate-dead-inserts
    eliminate-dead-code-aggressive

           Note this does not guarantee legal code. This option implies
           --skip-validation.

Note the latest about --skip-validation

What i and others did was try to pinpoint which of THOSE was necessary to run superposition, and it seems as 3 shaders needs: `--inline-entry-points-exhaustive --skip-validation --eliminate-dead-code-aggressive`
Running eg. Unigine Valley/Heaven, the --skip-validation is not necessary.

Recap: Running --legalize-hlsl does ALL of the above optimizations, including --skip-validation, therefor you will not see that particular error mentioned earlier :)
pdaniell-nv commented 6 years ago

@doitsujin Regarding https://pastebin.com/6cmCE49u it appears we have a couple of issues with this. Our inliner didn't handle this well, which is why when you inlined it yourself it worked better. I'm not sure yet why our inliner had a problem, but I'll report it to our compiler team to investigate.

The other issue we have with that shader is we don't support OpImageQuerySizeLod or OpImageQueryLevels on non-sampled images. If these ops were used on the %16 image "OpTypeImage %float 2D 1 0 0 1 Unknown" they would have worked okay. Core Vulkan/SPIR-V doesn't really support levels with storage images; there is no way to access texels from Lod>0 through OpImageRead/OpImageWrite. Because of this there is probably limited utility in using OpImageQuerySizeLod or OpImageQueryLevels on non-sampled images.

In that shader, you can get it to work by making the following modification:

         %31 = OpLoad %16 %u0  # changed from %17 to %16
         %33 = OpImageQuerySizeLod %v2uint %31 %30
         %34 = OpLoad %16 %u0  # changed from %17 to %16
         %35 = OpImageQueryLevels %uint %34

That appears to fix the inlining crash too, so they're probably related.

I may investigate a SPIR-V spec fix for this. I think in unextended SPIR-V 1.2 that OpImageQuerySizeLod and OpImageQueryLevels should only be defined to work with sampled images. It looks like the SPV_AMD_shader_image_load_store_lod extension adds Lod support to storage images.

pdaniell-nv commented 6 years ago

@nairaner This looks like another case of OpImageQueryLevels being used with a storage image rather than a sample image. If possible, try to use OpImageQuerySizeLod and OpImageQueryLevels only with sampled images.

Actually, the CS_3e57aa98bdc58df0415b95c62877ef9e630eff9f.spv shader looks similar to https://pastebin.com/6cmCE49u.

SveSop commented 6 years ago

@pdaniell-nv I did some testing with Unigine Heaven, and found the two shaders i need to get things running. Take a look at : nVidia issues #64

pchome commented 6 years ago

Here is wine log for default Medium preset but Windowed on GeForce GTX 750 Ti 387.34. Looks normal as I see this benchmark for the first time. First scene starting with too dark environment but later all looks good. Min:11-15/Avg:18/Max:23-25 fps dxvk-superposition-wine.log

It's custom linux build w/ dll-redirect support and "auto legalizer", so I can run lancher with d3d11.dll and set redirects to d3d11_vk.dll only for superposition.exe. Then it runs. Otherwise launcher starts as black screen with no controls visible. Trick with replacing original dlls with dxvk "on the fly" works too.

Edit: opengl mode: dxvk-superposition-wine-opengl.log

doitsujin commented 6 years ago

@pdaniell-nv that makes sense, again thanks for the info. That looks like a problem in my implementation of the resinfo instruction (and possibly others) when used on image UAVs, which are mapped to non-sampled images.

pingubot commented 6 years ago

@doitsujin : thanks for also fixing that issue. Heaven still crashes here with the current dxvk build, so there is still minimum one shader issue left which @pdaniell-nv hopefully should be able to clarify with the 2 shaders which @SveSop provided. Those are the only 2 right now for Heaven on Nvidia Linux which still cause crashes.

SveSop commented 6 years ago

@pingubot Could you test if Superposition starts and runs until scene 9 with the 3 shaders attached in "working.tar.gz"? The broken ones i attach in "broken.tar.gz". I ran: spirv-opt --inline-entry-points-exhaustive --skip-validation --eliminate-dead-code-aggressive to get them working.

Just be aware that there is 1 or maybe more shaders that does not work tho, cos it will crash at "scene 9" when running benchmark or cinematic mode. broken.tar.gz working.tar.gz

SveSop commented 6 years ago

@pingubot Found the 4th one that will make Superposition run without crashing. Same deal "working.tar.gz" and "broken.tar.gz" for comparison. Superposition Preset: 1080p Medium

@pdaniell-nv This is with nVidia Linux driver 390.25 tho. And if you look at my earlier post with the screenshot Screenshot Its almost as its a thin layer of dust on everything. (Perhaps not that bad, since it looks like a lab from the 60'ies heh) broken.tar.gz working.tar.gz

SveSop commented 6 years ago

Have been so caught up with finding crashing shaders, that i havent really payed much attention to the logs.. But now that those 4 shaders get things running somewhat "ok", i saw the logs having quite a few: err: D3D11Device::CreateGeometryShaderWithStreamOutput: Not implemented Guess something that will be implemented at some point?

err:   D3D11Device::CreateBuffer: D3D11_BIND_STREAM_OUTPUT not supported
err:   D3D11DeviceContext::SOSetTargets: Not implemented

More of the same i guess. Attaching logs for giggles and laughs :) superposition_d3d11.log superposition_dxgi.log

oscarbg commented 6 years ago

@doitsujin sorry for lack of clarity for Intel HD graphics working I disabled depth bounds which upped support to feature level 10.1 but for 11.0 only needed disabling shaderStorageImageReadWithoutFormat as shaderStorageImageWriteWithoutFormat is already supported (see: https://vulkan.gpuinfo.org/displayreport.php?id=2600#features) the other one I have to disable is variableMultisampleRate (as you can see from linked report is missing on Intel Mesa aka Anvil driver).. anyway I still have working antialiasing on some apps like on Witness on iGPU so what you are using variableMultisampleRate for? good news is that variableMultisampleRate is supported on Windows Intel HD graphics closed source Vulkan driver (see: https://vulkan.gpuinfo.org/displayreport.php?id=2529#features) so should have to disable only depth bounds and shaderStorageImageReadWithoutFormat there.. it also shows that Mesa devs can implement variableMultisampleRate support on Intel HD graphics.. perhaps do a request on mesa-dev mailing list? I mean is not like depth bounds test support that seems is not supported in HW by current Intel HD graphics.. but somewhat sad is I tested also Intel HD graphics on Windows closed source Vulkan driver disabling this extensions but I'm getting a hang somewhere inside their driver igvk64.dll I think.. so have to debug more.. will report success and some patches if found the root of the cause..

@SveSop thanks for the detailed explanation of why I was not getting validation errors when running full --legalize-hlsl command good to know it disables validation..

oscarbg commented 6 years ago

@pdaniell-nv @doitsujin I think I'm on to something interesting with DXVK+NV using ComputeMark2.. Please test these repro: ComputeMark2repro.zip TLDR: executing ComputeMark MandelS test DXVK generates valid SPIR-V files and no errors are returned by Nvidia Vulkan driver regarding creation of compute or graphic pipelines, yet we see blank screen.. on AMD GPU all work nice..

it's ComputeMark2 with DXVK precompiled and included spir-v files of all benchmarks.. I'm using the latest dxvk including all latest 2 NV fixes: [dxbc] Use correct image query instructions for UAVs [dxvk] Only emit depth image types for 2D and Cube images Please run either ComputeMark.exe or runanddump.bat file that will also redump files..

it's very easy from Windows10 just run ComputeMark.exe then from "Burn test demo" select the individual test you want and select Go button: MandelS and Fluid3D are of interest.. a window will be created for running the test but this window will remain black during all the test.. at least for me on Nvidia GTX 970+390.65 driver..

the thing is all spir-v files produced by DXVK are now valid according to spirv-val (i.e. doesn't require running spirv-opt).. you can check all SPIR-V files on spirvshadersnv folder and from that folder a valshaders.bat will run spir-val for each spir-v file without any warnings/or errors..

even one test (MandelS) doesn't report any errors creating pipelines (see ComputeMark_d3d11mandels.log captured with "DXVK_LOG_LEVEL=debug").. still the window obtained is black..

the other interesting test is Julia3D this produces 2 DxvkComputePipeline errors "err: DxvkComputePipeline: Failed to compile pipeline" altough as said SPIR-V files are now valid according to spir-val.. screen renders black..

of course all this run and render OK on AMD Vulkan driver on Windows..

as said the more interesting test is MandelS as it doesn't even create any error creating graphics or compute pipelines..

pdaniell-nv commented 6 years ago

Thanks @oscarbg. I ran the new ComputeMark2repro.zip. The shaders that have OpBitFieldInsert found a bug in our compiler in that we didn't handle the "Offset" and "Count" operands being anything other than signed int. This causes a crash in the compiler. I fixed it locally to handle both signed and unsigned and I was able to get all ComputeMark.exe tests to compile without fault in our compiler.

Unfortunately, all tests produce a black window. I didn't get a chance to investigate why this is. My guess is maybe there is a mismatch between the shader and the pipeline layout such that some resources aren't being picked up.

Have you tried running this with Vulkan validation enabled? Perhaps tweak DXVK to emit integers for the "Offset" and "Count" operands of OpBitFieldInsert and OpBitField{S,U}Extract and see what the Vulkan validation says.

In the meantime we'll work on getting the compiler fixes for the driver released.

pdaniell-nv commented 6 years ago

FYI, I force enabled validation layers locally and there are a few errors being output. One of them which may give a clue is this one:

DS(ERROR / SPEC): object: 0x452 type: 23 msgNum: 364905100 - vkUpdateDescriptorSets() failed write update validation for Descriptor Set 0x452 with error: Write update to descriptor in set 1106 binding #3 failed with error message: Attempted write update to image descriptor failed due to: ImageView (77) with usage mask 0x23 being used for a descriptor update of type VK_DESCRIPTOR_TYPE_STORAGE_IMAGE does not have VK_IMAGE_USAGE_STORAGE_BIT set.. The spec valid usage text states 'If descriptorType is VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, or VK_DESCRIPTOR_TYPE_INPUT_ATTACHMENT, the imageView and imageLayout members of each element of pImageInfo must be a valid VkImageView and VkImageLayout, respectively' (https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#VUID-VkWriteDescriptorSet-descriptorType-00326)

doitsujin commented 6 years ago

Edit: The validation errors actually seem to be caused by an application bug, it's trying to use a depth-stencil image with a UAV but never sets the appropriate D3D11 bind flags. Might call for an application-specific workaround.

oscarbg commented 6 years ago

@pdaniell-nv thanks for giving an inspection to the repro and glad to help NV fix some bugs.. to be clear I'm not a dev of DXVK only pointed you to issues using it on NV GPUs vs on AMD.. @doitsujin sad it's an app bug.. thanks also for giving some of your time to investigate the cause.. seeing your findings and if not much work would be glad if you implement that workaround.. primary reason is maybe is might not be the unique D3D11 app doing it badly but working OK on D3D11 drivers.. also strangely enough this app works correctly on Wine D3D11->OGL wrapper on AMD and NV GPUs if I recall correctly so seems in some way OpenGL is more forgiving or Mesa devs also ignore/set some flags independent of what the app says..

doitsujin commented 6 years ago

@oscarbg OpenGL simply does not have explicit usage flags, so you can't get them wrong in the first place.

pingubot commented 6 years ago

@pdaniell-nv : Many thanks. Can you also take a look at the shaders which @SveSop provided which crash the driver in unigine heaven and superposition ? They both work with the amd drivers.

jozefkucia commented 6 years ago

Edit: The validation errors actually seem to be caused by an application bug, it's trying to use a depth-stencil image with a UAV but never sets the appropriate D3D11 bind flags. Might call for an application-specific workaround.

I think it's a DXVK bug. CreateUnorderedAccessView() should fail if D3D11_BIND_UNORDERED_ACCESS is not set. There is probably a test for this behavior in Wine conformance tests.

pdaniell-nv commented 6 years ago

@pingubot Are you referring to https://github.com/doitsujin/dxvk/issues/70#issuecomment-366338851?

pingubot commented 6 years ago

@pdaniell-nv , yes, 70 and 74+75

doitsujin commented 6 years ago

I figured out what was causing the ComputeMark issue, the application wants to use the swap chain image with an unordered access view, which was not handled properly. It's the first time I see an application do that.

Fixed in 8813ff979a49d2ae0ae3e6d1a88ad0ac12d48993.

oscarbg commented 6 years ago

@doitsujin aargh.. first thanks to all @jozefkucia @pdaniell-nv and for your effort.. I feel somewhat guilty now.. I remember reading that many many years ago by the guy who implemented that.. really I vaguely remembered that this app "rendered right from the RWTexture it use to simulate" so I already have in my mind some though that ComputeMark was a "special" D3D11 to test so I insisted you to fix support it the same way a year ago already insisted Wine D3D11 devs to make it work.. I have been spending some time to refind it.. and so here, just for completeness, let's point to original individual demos (with source code!) done by Voxilla very shortly (within 1 month) D3D11 drivers started appearing for AMD 58xx series.. (circa end 2009) (maybe some of you were kids still by then :-)) : users.skynet.be/fquake/ here you can find also some demo not included in ComputeMark: 3D waves simulator (DX11) tough I expect it to work.. and now I see under "DX11 DirectCompute Mandelbrot and Julia viewer" on this site, a DX11 feature level DX10 version: http://users.skynet.be/fquake/MandelDX11_10.zip and note this:

DX10 support In order to support DX10 GPUs the DX11 feature level DX10 version should be used. It will try to make use of compute shader 4 instead of compute shader 5. This requires an additional pass with a pixel shader to copy compute shader output to the screen.

this demo should work now on Intel HD 530 Mesa driver now that you disabled depth bounds requeriment so gonna give it a try..