IIDX 9th/10th styles - Some videos (but not others) are stretched

[JeffPaine4890]

(doesn't look like there's a template for a bug report on Github? I just copied+pasted from another issue)

Summary Some videos are stretched on 9th/10th style, but not all videos.

Expected behavior Videos should not be stretched.

Current behavior Videos are stretched.

Detailed Description I haven't done a detailed accounting of which videos are affected, I just happened to notice it while playing era (nostalmix). I tried playing another song with a person in it for the purposes of taking screenshots to make it more obvious, but that song didn't appear to have a stretched video (Glorious Days).

Setting the flag to fix stretched videos does nothing in this case. (I do need this set for IIDX RED through DistorteD though)

The screenshot is taken running fullscreen at 1920x1080. I've tried running windowed at 1920x1080 and at 640x480 and get the same issue in both instances. (My computer doesn't want to run IIDX fullscreen at 640x480 for some reason so I was unable to test that)

Steps to reproduce

1. Start the game
2. Pick Free mode
3. Play era (nostalmix)
4. Notice the video is stretched

Context (Environment) Bemanitools version(s) affected 5.43 Game(s) and version(s) affected IIDX 9th style JAH and 10th style JAE

Command line arguments Nothing other than the defaults in gamestart.bat

APIs used None

OS version Windows 11 Pro Canary 25324

Hardware specs CPU: Intel Core i7-12700H, 2300 Mhz RAM: 16 GB GPU: NVIDIA GeForce RTX 3060 Laptop GPU, 6 GB Controllers/IO: Built-in laptop keyboard (10keyless)

Screenshots 9th style (10th style looks the same): IIDX RED (this is what it should look like):

[icex2]

Thanks for using the old template and good call that I need to add this back. Have you tried using the upscaling feature in bemanitools? The videos are decoded and rendered differently on 9 and iirc even 10. I know the BG video rendering code was updated on 11 and on 12, they switched to an external codec.

The code might also not be covering all edge cases because it works based on a rather bad heuristic to detect if a video needs to be stretched.

What's the song in question above? Looks like either era nostalmix or step mix judging by the bpm and "taq" on screen.

# Up-/downscale the back buffer's width. This does not change the game's rendering resolution but scales the final frame. Use this to target the native resolution of your monitor/TV, e.g. to avoid over-/underscan, bad image quality or latency caused by the monitors internal upscaler. 0 to disable this feature. Must be set in combination with the corresponding height parameter.
gfx.scale_back_buffer_width=0

# Up-/downscale the back buffer's height. This does not change the game's rendering resolution but scales the final frame. Use this to target the native resolution of your monitor/TV, e.g. to avoid over-/underscan, bad image quality or latency caused by the monitors internal upscaler. 0 to disable this feature. Must be set in combination with the corresponding width parameter.
gfx.scale_back_buffer_height=0

# Filter type to use for up-/downscaling the back buffer. Only used if scaling feature was enabled by setting the scaling width and height parameters. Available types: none, linear, point (refer to D3DTEXTUREFILTERTYPE  for explanation).
gfx.scale_back_buffer_filter=none

[JeffPaine4890]

Yeah, that song is era (nostalmix).

Those screenshots were taken using bemanitools' upscaler. Turning the upscaler off or changing the filter type makes no difference.

[icex2]

Ok, good to know. Can't give you any ETA on when I ever get around to that. There might be project opportunities where I see this a valuable addition and pick it up.

However, if anyone else reads this and wants to take a look, here is a pointer where to get started: in the iidxhook-util/d3d9 module, there is the iidxhook_util_d3d9_iidx11_to_17_fix_uvs_bg_videos function that handles all the video stretching/fixes based on a rather jank heuristic. Code should have sufficient comments explaining each case. However, ask away questions here if you have any. Some videos might have used a slightly different surface, so the numbers don't align with the existing cases. Adding some debug prints in here and running the affected songs might already give you some ideas if this part applies and if another case must be added with slightly different values. Otherwise, there could be other logic/render calls creating the surface different for some videos which are not trapped there yet. This requires checking the code for different render paths. A good entry point on which functions are called and understanding the entire render pipeline on a d3d9 level is using apitrace

[JeffPaine4890]

Okay taking another quick look at it, it seems pre-4th style videos are the only ones affected (including pre-4th generic videos used by songs from later styles). I'm quite certain this has to do with those videos being in 176x240 resolution as opposed to all the videos from 4th and later which are 240x240. Maybe the game is stretching the videos, then bemanitools is stretching them again? Or possibly the game is stretching them too much and bemanitools is not detecting this to fix it?

If I have time later I'll play around with the code to see if I can get it working myself, but this isn't exactly a priority for me either :P

[JeffPaine4890]

After a little more digging it seems there's a flag in the song data that determines if the video should be stretched or not. As an example, replacing 02ho.4 with 0900.4 will still result in a stretched video if you play, for instance, .59, but when you play lower world the video still looks normal. Likewise, going the other way will result in lower world playing the 02ho video without stretching. (Possibly also the game has its own list of which videos should be stretched or not)

[JeffPaine4890]

I've made some progress on this, all it will take is some slight modifications of the iidxhook_util_d3d9_iidx11_to_17_fix_uvs_bg_videos function since the 9th/10th texture positions aren't covered by what's already there

I won't finish it tonight but at least I know exactly how to fix it now, so probably within the next few days I'll have a pull request for this

[icex2]

I've made some progress on this, all it will take is some slight modifications of the iidxhook_util_d3d9_iidx11_to_17_fix_uvs_bg_videos function since the 9th/10th texture positions aren't covered by what's already there

I won't finish it tonight but at least I know exactly how to fix it now, so probably within the next few days I'll have a pull request for this

Thanks for taking the time and looking into this. Once you consider our changes working and want to get some feedback, just open a PR and we continue the discussion there.