Arcanum: Of Steamworks and Magick Obscura issue disappearing ending slides

[Eselter]

When we complete game or die, game displays slides (bitmaps). Game should displays the slides until player press something but very often this slides only appear for split second.

Log from debug version: ddraw.log.zip

System: Windows 10 x64 Game: Arcanum with patch UAP2.0 DDrawCompact: experimental/test release from 2020-04-12

In my observations in latest experimental version is better than on 0.2.1. When I run game with slower ddraw wrapper this doesn't happen but game runs not that smooth.

[Eselter]

Problem exists in latest release from 2020-04-19. Arcanum 2020-04-20 00-42-09-03.mkv.zip

[narzoul]

I could not yet reproduce the issue with either the default GOG install or with UAP 2.0 added. From googling around it seems to be an old problem (from way before DDrawCompat), so at least I think it's not something broken in my code.

It is supposed to be reproducible by just clicking on credits in the main menu also. Can you reproduce it that way? I tried watching the death slide like a dozen times too but it works correctly every time. For both cases, there is a slow fade out/fade in effect when the slide (or credits) appears, and then again when it disappears. But this seems to be missing for you too based on your video. I guess the two issues might be somehow related.

If you can't reproduce it with the credits, then could you attach your save game also? I don't really have an end game save at hand. If credits are broken too then there is probably no need for this.

From the logs, it seems you have an Nvidia GPU. Maybe the issue is specific to that, but unfortunately that's something I can't test at the moment. I didn't see anything wrong in the logs though. Can you test your integrated GPU too if you have one?

[narzoul]

Actually, I think I figured it out after comparing your logs with mine. For the fade effects, the game uses repeated calls to SetGammaRamp, which on my AMD/Intel drivers always waits for v-sync, while on your driver it doesn't, so the whole effect finishes too quickly. I'm not sure how this ties in to input handling, but if I disable the SetGammaRamp calls, then I also get the credits rolling by too fast (multiple slides per button press, sometimes none of them show up at all), and the death slide occasionally skips too.

I'll try to figure out a proper solution tomorrow, my attempts to introduce a v-sync wait in SetGammaRamp have produced some undesirable side effects so far.

[Eselter]

Yeah in credits (arcanum module) I see sometimes 5, sometimes 6 screens. In polish version of game there should be 11 credits screens (I check this on windowed mode). When I play in windowed mode all is fine, but game seems to freeze for few seconds when loading slides.

Below save before ending in module Lost Dungeon of Souls (click on the door). LostDungeonofSouls save before ending.zip

And thank you for looking at this :)

[Eselter]

I don't have iGPU in my CPU but I test this in very old notebook with Radeon HD 4570 and I have the same issue.

Log from notebook on credits screen (ddraw from 2020-04-19 in debug mode). I see 8 of 11 slides. ddraw debug credits notebook with amd gpu.zip

PS: When game run slower (ex. using debug ddraw) is better (less often screen disappear too fast),

[Eselter]

I also try force v-sync in nVidia control panel for Arcanum and running game with different compatibility settings but without success.

[Eselter]

I add sleep (50ms) and now is working very nice :) But this is only workaround not solution ;/

#include "DDraw/DirectDrawGammaControl.h"
#include "DDraw/RealPrimarySurface.h"
#include "DDraw/Surfaces/PrimarySurface.h"

namespace
{
    bool isPrimaryGamma(IDirectDrawGammaControl* gamma)
    {
        Compat::Log() << "Eselter test 1";
        return CompatPtr<IDirectDrawSurface7>::from(gamma) == DDraw::PrimarySurface::getPrimary();
    }

    HRESULT STDMETHODCALLTYPE getGammaRamp(
        IDirectDrawGammaControl* This, DWORD dwFlags, LPDDGAMMARAMP lpRampData)
    {
        Compat::Log() << "Eselter test 2";
        if (0 != dwFlags || !lpRampData || !isPrimaryGamma(This))
        {
            Compat::Log() << "Eselter test 2-1";
            return DDraw::DirectDrawGammaControl::s_origVtable.GetGammaRamp(This, dwFlags, lpRampData);
        }
        Sleep(50);
        return DDraw::RealPrimarySurface::getGammaRamp(lpRampData);
    }

    HRESULT STDMETHODCALLTYPE setGammaRamp(
        IDirectDrawGammaControl* This, DWORD dwFlags, LPDDGAMMARAMP lpRampData)
    {
        Compat::Log() << "Eselter test 3";
        if ((0 != dwFlags && DDSGR_CALIBRATE != dwFlags) || !isPrimaryGamma(This))
        {
            Compat::Log() << "Eselter test 3-1";
            return DDraw::DirectDrawGammaControl::s_origVtable.SetGammaRamp(This, dwFlags, lpRampData);
        }

        return DDraw::RealPrimarySurface::setGammaRamp(lpRampData);
    }
}

namespace DDraw
{
    void DirectDrawGammaControl::setCompatVtable(IDirectDrawGammaControlVtbl& vtable)
    {
        Compat::Log() << "Eselter test 4";
        vtable.GetGammaRamp = &getGammaRamp;
        vtable.SetGammaRamp = &setGammaRamp;
    }
}

[Eselter]

Also adding 1 ms sleep (instead 50ms sleep in above workaround) in "bool isPrimaryGamma(IDirectDrawGammaControl* gamma)" also look really nice and smooth.

[Eselter]

Also I notice that fraps can't record this fade out effect (OBS the same also bandicam). I need find better tool for recording (GeForce Experience tool not working with Arcanum for some reason).

[Eselter]

My test build: ddraw 1ms sleep in setGammaRamp.zip

[Eselter]

I don't know why but I don't find any method to record fade out effect on PC. Even on virtual machine or remote screen this effect is not visible. Probably this can be recorder only with external recorder. Below video from my phone with ddraw with 1ms sleep in setGammaRamp. In real looks better is really smooth. Vid 20200421 174309-1.zip

[narzoul]

Well, I googled a lot and I'm no closer to finding out what is the expected behavior of SetGammaRamp. The DirectX 7 docs are silent on the issue, but DX8 and 9 docs do explicitly mention that it doesn't wait for v-sync, which is at least not true on both my current Intel and AMD drivers.

An old tutorial from 2000 that implements fade effects in DirectDraw with SetGammaRamp apparently uses Sleep(1) because "the routine was a little too fast so this slows it down a bit....": https://www.gamedev.net/reference/articles/article998.asp Yet someone else in 2003 observed that it's "too slow" because it waits for v-sync on his Geforce 4: http://www.verycomputer.com/289_2f29870b429e923b_1.htm

So it looks like the behavior has been inconsistent among different drivers for a long time. I guess the intended fade effect depends on what hardware/drivers the game was tested with. Since there doesn't seem to be any speed limit in the game itself, I'm leaning towards that it depended on waiting on v-sync, otherwise they would have probably ran into the same issue that the fade is too fast and can even skip many slides.

That said, I'd be cautious with adding that much delay to drivers that aren't currently waiting on v-sync, because other games might have different expectations and it could end up making those run too slow when using gamma control for effects. So in the end I'll probably also end up adding just a 1 ms sleep as a compromise, in case the driver doesn't already wait at least that long.

This will result in the fade duration being inconsistent on different drivers, but fixing that would require emulating the gamma ramps via a pixel shader. Which is pretty difficult to implement in Direct3D 7, because it doesn't expose any interfaces for using shaders. It would have to be hacked in via other means, which is something I eventually plan to do anyway for different reasons, but that is probably very far in the future.

As for recording the fade effect: yes, I assume most recorders don't take into account the gamma ramp levels and just record the primary surface buffer contents (which is not affected by gamma changes). You'd have to find something that can track and record the gamma adjustments also, but I don't know of any recording software that can do that.

[Eselter]

So it looks like my workaround using 1ms sleep in setGammaRamp is not that bad after all ;)?

[Eselter]

#include "DDraw/DirectDrawGammaControl.h"
#include "DDraw/RealPrimarySurface.h"
#include "DDraw/Surfaces/PrimarySurface.h"

namespace
{
    bool isPrimaryGamma(IDirectDrawGammaControl* gamma)
    {
        return CompatPtr<IDirectDrawSurface7>::from(gamma) == DDraw::PrimarySurface::getPrimary();
    }

    HRESULT STDMETHODCALLTYPE getGammaRamp(
        IDirectDrawGammaControl* This, DWORD dwFlags, LPDDGAMMARAMP lpRampData)
    {
        if (0 != dwFlags || !lpRampData || !isPrimaryGamma(This))
        {
            return DDraw::DirectDrawGammaControl::s_origVtable.GetGammaRamp(This, dwFlags, lpRampData);
        }
        return DDraw::RealPrimarySurface::getGammaRamp(lpRampData);
    }

    HRESULT STDMETHODCALLTYPE setGammaRamp(
        IDirectDrawGammaControl* This, DWORD dwFlags, LPDDGAMMARAMP lpRampData)
    {
        Sleep(1);
        if ((0 != dwFlags && DDSGR_CALIBRATE != dwFlags) || !isPrimaryGamma(This))
        {
            return DDraw::DirectDrawGammaControl::s_origVtable.SetGammaRamp(This, dwFlags, lpRampData);
        }

        return DDraw::RealPrimarySurface::setGammaRamp(lpRampData);
    }
}

namespace DDraw
{
    void DirectDrawGammaControl::setCompatVtable(IDirectDrawGammaControlVtbl& vtable)
    {
        vtable.GetGammaRamp = &getGammaRamp;
        vtable.SetGammaRamp = &setGammaRamp;
    }
}

Build with windows SDK 10.0.18362.0 ddraw 1ms sleep fix without additional logs.zip

[narzoul]

I did some further testing. Some old forum posts mentioned that this issue can be fixed by disabling DirectDraw hardware acceleration. I guess this is similar to forcing the HEL device to be used with the ForceDirectDrawEmulation shim. The relevant effect of this is that the HEL device does not expose any gamma control capabilities (which would require hardware support to work).

So I experimented with something similar by simulating disabled gamma caps in the driver. In this case, the game does not use the IDirectDrawGammaControl interface at all. However, it still waits for exactly 2 seconds whenever a fade in or fade out effect is supposed to play. More precisely, if I press a key to change slides during credits, it gets the WM_KEYDOWN message, waits for 2 seconds by polling timeGetTime (this is when the fade out should play), updates the screen, then waits for 2 seconds again (placeholder for fade in), and only then it processes the WM_KEYUP message. I think this is more "proof" that the fade effect was supposed to be fairly slow. With gamma control, the game executes 48 gamma changes per fade, which at 60 Hz refresh rate (with v-sync) takes ~800 ms. Just 1 ms sleep per gamma change would make it too fast in comparison.

I think I'll rather go with enforcing v-sync on gamma changes even for those drivers that currently don't do that, after all. This will give a uniform behavior across drivers and I think it will also match how the fade effect was intended to work in Arcanum. If this causes any problems in other games, I'll revisit the issue again. Though in that case only a config option would provide a proper solution.

[Eselter]

Sounds like a good plan. I waiting for new test release ;)

[narzoul]

Fixed in the latest experimental release.

[Eselter]

Thanks, Fix working great on PC and notebook :)