[d3d11] NieR Replicant ver.1.22474487139... prerendered media fails to play video

aurieh commented 3 years ago

When attempting to play a prerendered "movie" file, the screen turns black, no video is played (however audio plays fine) and DXVK logs an error. The files utilize a proprietary format, but when its 24-byte header is removed we get an ASF container.

$ ffprobe MOVIE_ATRACT_GESTALT.asf
[snip.]
Input #0, asf, from 'MOVIE_ATRACT_GESTALT.asf':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    encoder         : Lavf58.20.100
  Duration: 00:03:07.09, start: 0.000000, bitrate: 25345 kb/s
    Stream #0:0(eng): Video: wmv2 (WMV2 / 0x32564D57), yuv420p, 1920x1080, 25000 kb/s, 30 fps, 30 tbr, 1k tbn, 1k tbc
    Stream #0:1(eng): Audio: wmav2 (a[1][0][0] / 0x0161), 48000 Hz, 2 channels, fltp, 128 kb/s

The issue can be reliably reproduced by waiting at the "press any button" screen for about 30 seconds, at which point the game will attempt to play (one of?) its trailer(s) (data/movie/MOVIE_ATRACT_{REPLICANT,GESTALT}.arc). According to https://github.com/ValveSoftware/Proton/issues/4777#issuecomment-828716546, this is also an issue when using DXVK on Windows. It's a major progression roadblock as a prerendered cutscene plays at the midpoint of the game.

After a quick search, I was able to find issue https://github.com/doitsujin/dxvk/issues/1676, which seems to be related (in that this is also caused by missing YUV support in the d3d11 implementation).

Software information

Game: NieR Replicant ver.1.22474487139... Settings do not affect the ability to reproduce this issue.

System information

GPU: Radeon RX 580 8GB
Driver: amdgpu (kernel 5.11.6), RADV, mesa 20.3.4
Wine version: 6.7-GE-1 with a couple patches described in https://github.com/ValveSoftware/Proton/issues/4777#issuecomment-829523695
DXVK version: 1.8.1

Apitrace file(s)

Unfortunately, the game doesn't run under wined3d and I do not have a Windows setup to run apitrace on, not one that has a real GPU attached at least.

Log files

d3d9.log: not applicable
d3d11.log: https://gist.github.com/aurieh/95d00c05d880853c804d14cef5ccac84
dxgi.log: https://gist.github.com/aurieh/fb6b2caad9c85987ac7abd1ed321adbf

jpgay commented 3 years ago

Apitrace file (massive) for attract movie playback from my Windows 10 VM with a 1050ti attached.

doitsujin commented 3 years ago

Looks like the game requires NV12 support. That's a huge issue since DXVK does not support multiplanar formats at all, and adding it now is going to be extremely hard. It also doesn't help that there is hardly any documentation on how to actually use that format in D3D11 (especially on how uploads and image copies work etc).

Is there a Proton build with the required patches available to test this with? Regular Proton obviously just crashes.

Note that Windows has a different problem since Microsoft's mfplat implementation requires support for D3D11 Video APIs. We cannot support those.

MagZu commented 3 years ago

Looks like the game requires NV12 support. That's a huge issue since DXVK does not support multiplanar formats at all, and adding it now is going to be extremely hard. It also doesn't help that there is hardly any documentation on how to actually use that format in D3D11 (especially on how uploads and image copies work etc).

Is there a Proton build with the required patches available to test this with? Regular Proton obviously just crashes.

Note that Windows has a different problem since Microsoft's mfplat implementation requires support for D3D11 Video APIs. We cannot support those.

Check out the build this guy made https://github.com/ValveSoftware/Proton/issues/4777#issuecomment-829859050 (not sure if link works but user jbgay in this issue: https://github.com/ValveSoftware/Proton/issues/4777) He made a proton build with a patch reversed which does not crash NieR while the prerendered cutscenes are playing.

doitsujin commented 3 years ago

Starting to make some real progress; the videos play on Windows (although it should also work with a Proton build that has the required mfplat patches; even when using NV12 images). Branch is https://github.com/doitsujin/dxvk/tree/nv12, but there's still a lot of work (and research) to be done before this can be pulled into DXVK.

2021 05 19-19 59

doitsujin commented 3 years ago

Fixed in current master.

hotcapy commented 2 years ago

Hello, dear @doitsujin and users.

I still have a problem described here regardless of Wine or Proton build/version I use.

My game always crashes after loading screen when finish Act I cutscene should appear. Same problem on "Press any button" title screen.

I tried to use latest versions of Wine and Proton by GloriousEggroll and Tk-Glitch, run game via Lutris and Steam, install to Wine prefix "vcrun2019" and "mf" via Winetricks, add MS Media Foundation libraries via "mf-install" script by z0z0z, but nothing of it helped. And, of course, I am using latest DXVK built-in in Lutris or Proton.

Interesting thing is that when Special K and GShade are disabled game just immediately closes, but when they are enabled video plays just with sound but without any picture (black screen), Special K gives some critical error and nothing happens when cutscene ends (well, sound of cutscene ends, since I see nothing else), it just remaining in black screen stage forever until I stop game manually.

It would be perfect if someone can help me a little and tell me which exactly additional software should be installed in Wine prefix to make video playback in NieR Replicant work. I tried to figure it out myself by searching NieR's ID in "protonfixes" but suddenly my knowledge is still insufficiently for understanding it.

What else I need except "vcrun2019" or "mf" and where I can get it? Thank you for any answers and possible help in advance! :-)

doitsujin commented 2 years ago

Still works here with an old Proton-GE build (6.10-GE-1 to be precise), but yeah, otherwise it's complaining about a gstreamer plugin missing.

doitsujin / dxvk