bo3b / 3Dmigoto

Chiri's DX11 wrapper to enable fixing broken stereoscopic effects.
Other
779 stars 124 forks source link

Add automatic profile assignment from d3dx.ini #29

Closed bo3b closed 7 years ago

bo3b commented 8 years ago

There are nvapi calls to manage profiles, including for a given running process, whereby we could just apply a given profile that we read out of the d3dx.ini.

This would save the end-user from having to manually do those tweaks, and simplify our setups and game install guides. Also would ensure that the user didn't 'miss a step'.

DarkStarSword commented 8 years ago

This has been on my TODO list for some time. There's a few things to consider here:

bo3b commented 7 years ago

@helifax Are there a certain subset of profile tweaks that are valuable? Or is it more a requirement to have a full profile to use?

Based on the API, it looks like we can do single parameter tweaks to a profile, so in theory we could fetch the global profile, then tweak a game based on settings from d3dx.ini, without having to include an entire profile.

DarkStarSword commented 7 years ago

My plan at the moment is to add a new [Profile] section containing any settings that we need. The preferred usage would be something like:

[Profile]
StereoProfile = 1
StereoFlagsDX10 = 0x00004000
StereoConvergence = 0.5
Comments = "Far Cry Primal community fix. Please disable motion blur."
Disable2DD = 1
2DD_Notes = "Compatibility mode is enabled. To use the fix, please disable it with Ctrl+Alt+F11"

When the DLL is initialised it will load the current profile, decrypt any internal settings (the code for this is in 3DMigoto now - it logs the profile on launch with all settings decrypted) and compare them to the settings in the ini file. If any differ it will attempt to elevate privileges and write the new settings, creating a new profile if one does not exist. Settings will always be written as decrypted user settings - we will never make any attempt to write settings as predefined or internal.

While not preferred, for convenience I'm also thinking about allowing the [Profile] section to accept input as saved from Geforce Profile Manager - that is, allowing "Setting ID_0x" and "StringSetting ID_0x" on the LHS and allowing encrypted settings on the RHS, which we will decrypt and store as unencrypted user settings. I would ignore any encrypted string settings though, as there is a high probability those would have been corrupted thanks to the garbage encoding they use.

Some of those settings are things that the user may well want to override themselves (StereoConvergence, Disable2DD, StereoMemoEnabled come to mind), and in that they would not want us to try to write the profile on every launch. I'm thinking about adding a list of specific settings in 3DMigoto that we should allow to be user settings that don't match our profile. If they are missing or predefined we would still write them, and if we need to update the profile for any other reason we would still write them (the later is to allow for the possibility we might want to update these in a new version of the fix - adding a version number to the Comments setting would then allow us to force an update of the profile writing these new settings). Alternatively these could be in a second section like [ProfileUser].

DarkStarSword commented 7 years ago

@helifax - I doubt the API is intended to allow predefined values to be set, but regardless the reason that CM profile settings seemed to be problematic is because they are encrypted, and writing those values back to the driver would treat them as decrypted and effectively corrupt them. Take a look here: http://wiki.bo3b.net/index.php?title=Driver_Profile_Settings#Important_note_about_encoding_of_.22Internal.22_Settings

helifax commented 7 years ago

Yeah, maybe it doesn't but I remember reading somewhere in Nvapi.h that you can/should be able to do it. But maybe, the ID is wrong:-s

The way I did for my wrapper is like this: You will always need 1 setting that is mandatory:

Stereo TextureEnable: 0x4208B6C. This one is actually the Stereo Texture that we use in the shaders and is the one that makes 3D Vision Automatic work. Without it you will get flat 3D.

There is also the setting that allows 3D Vision to run in Window mode, but is DX9 only, so we can skip this one.

In my wrapper basically I use these settings and then I load a file and read it. I then populate all the ID with the values. This way, I can simply export it from NvInspector, put it in a file and next to the DLL.

I think we should do this one here as well. For example some profiles require the DX10DSCBNumber & DX10VSCBNumber (TitanFall 2) or different SLI bits and SLI modes, Vsync Modes and so on. So, I highly recommend this approach. Is easy to maintain as well afterwards. If you want to see the code from my wrapper related to Nvidia Profiles is here: https://github.com/helifax/OGL-3DVision-Wrapper/blob/master/src/src/NvApiProfileSettings.cpp There is also an updated version of this one with more Settings, that I can share;) as is not part of the public repository;)

It was all written based on the Nvidia Best Practices PDF. (Enumerating the profiles, find the profile or creating a new one, etc) Please let me know if I can be of any help:)

DarkStarSword commented 7 years ago

Stereo TextureEnable: 0x4208B6C. This one is actually the Stereo Texture that we use in the shaders and

The stereo texture we pass to shaders is one that we manufacture ourselves using the stereo signature and is nothing to do with this setting - I'm not sure if that is what you mean. The driver doesn't actually pass a stereo texture, it passes a stereo constant buffer (cb12 in DX11) or stereo constant register (c255 in DX9) into any vertex shaders it modifies that is separate from our stereo texture, but we don't rely on it because it won't be passed into shaders that the driver did not modify, and the values it contains are x=-separation*convergence, y=separation, so the maths to apply the formula is a little different.

is the one that makes 3D Vision Automatic work. Without it you will get flat 3D.

We have details of that one from nvidia - in high level terms it controls the heuristics to decide which render targets and depth buffers are automatically stereoised vs which are left mono, in what situations the driver will run the shaders twice, and in what situations the driver will apply the stereo correction formula, as well as a few game specific hacks. A couple of publicly known bit definitions are listed on the wiki, but bo3b and I have a fairly complete list from nvidia.

We generally don't want to set this as the default value of 0x23 (decrypted) is appropriate for most games. What we don't want to do is set it to 0x00000000 or 0xffffffff, both of which disable 3D for different reasons (well... I can think of a reason I might actually do that in a few more features time ;-)

There is also the setting that allows 3D Vision to run in Window mode, but is DX9 only, so we can skip this one.

If that's STEREO_STEREOPROFILE (the official name for 0x701EB457, but I agreed with the author of nvidia profile inspector to call it StereoProfile for consistency with the other names extracted from the driver) you are talking about we need it in DX11 as well since it is a prerequisite for a lot of the other settings to have any effect, as well as allowing convergence to be saved, etc. Notably, this setting is special in that the value doesn't seem to matter for most of the code paths it enables in the driver, only that it is present... but that is technically a bug - it should be set to 0x00000001 in all cases (if you see any other value you have not decrypted it).

In my wrapper basically I use these settings and then I load a file and read it. I then populate all the ID with the values. This way, I can simply export it from NvInspector, put it in a file and next to the DLL.

I'm not actually keen on that idea as it is not self documenting and replaces all the settings, whereas I much prefer understanding what we are changing, and only changing what we need. I want a priority on documenting the most common options we need by name in the d3dx.ini template with common examples, and I want floating point settings to be written as... floating point values. And if a newer driver ships with better SLI bits than the initial release and all our fix cares about is StereoFlagsDX10 as we usually do I'd rather leave the SLI bits alone and only change them if we know we need to. I'm not saying we couldn't support loading a .nip file, but I am saying I would strongly discourage its use.

Plus, if the .nip file was generated with an old version of nvidia inspector it won't have the settings decrypted as that code was only added earlier this year, guaranteeing that we corrupt those settings. (I'm not positive the new versions use decrypted values in the .nip files either - I was happy once the developer implemented decryption based on my code so that the UI would display the right values and didn't look further).

I think we should do this one here as well. For example some profiles require the DX10DSCBNumber & DX10VSCBNumber (TitanFall 2) or different SLI bits and SLI modes, Vsync Modes and so on.

That should work this way as well - in addition to the stereo settings names my code can also use the names provided by the driver, or the hex setting ID.

If you want to see the code from my wrapper related to Nvidia Profiles is here:

I might borrow one or two things but I'm not really worried about that - the API is simple and I have all the docs from nvidia and most of it implemented already (and I've done this all before in my Stereo Photo Cropping Tool... in Python no less) - plus I really want to handle the encrypted settings properly, because they are the number one reason that we have so much confusion around all the settings to date (although, annoyingly there is no indication of which settings are encrypted through the main DRS API - for that I'm doing the same thing that nvidia profile inspector does and using the undocumented NvAPI_DRS_SaveSettingsToFileEx call, which writes out a file in the same format as Geforce Profile Manager).

How did you handle the admin requirement to write the settings? Do you need the whole game to be run as admin? I don't want to require the game be run as admin because I can't be sure the game will be well behaved (i.e. doesn't write a file as admin that it won't be able to access later as a limited user, doesn't open a network socket for multiplayer that now allows hackers admin access instead of limited user access (because games have never had security vulnerabilities in their netcode cough cough unreal tournament cough metasploit cough), so I'm going to use a helper program for that part (for now I'm using rundll as that helper program, with the caveat that doesn't work if there are spaces in the filename and no short paths, so in that case I'll just copy the DLL off to the temp directory first. Later I'll probably ship or embed our own helper program for a better user experience, but this works).

helifax commented 7 years ago

Yes, the full game requires admin rights in Windows 8/10 for the Profile to be updated. How you will implement it is up to you guys:) I just wanted to express my POV. :)

Don't know anything about "NvAPI_DRS_SaveSettingsToFileEx" though as I didn't see this one being referenced anywhere :(

DarkStarSword commented 7 years ago

Check out the source code to nvidia profile inspector if you want it's ID (and the ID for the corresponding NvAPI_DRS_LoadSettingsFromFileEx), which you can pass to nvapi_QueryInterface to get a function pointer. The arguments are the same as the non-Ex versions ;-)

bo3b commented 7 years ago

My plan at the moment is to add a new [Profile] section containing any settings that we need. The preferred usage would be something like:

[Profile]
StereoProfile = 1
StereoFlagsDX10 = 0x00004000
StereoConvergence = 0.5
Comments = "Far Cry Primal community fix. Please disable motion blur."
Disable2DD = 1
2DD_Notes = "Compatibility mode is enabled. To use the fix, please disable it with Ctrl+Alt+F11"

I like this idea. This seems like a good way to handle the profile settings in a minimalist change fashion.

My preference is also to not have a separate file for profile information, as part of the goal of smaller, less confusing installs. Since we can add this to the .ini file, that seems like the right spot for it, especially if it's possible to do single line/option changes.

Less sure about the need for admin privileges. Might be worth some experiments to see if that's strictly necessary, or if we can override live, without saving. Unlikely, but maybe they get picked up by a Ctrl-F7.

DarkStarSword commented 7 years ago

This feature is basically complete now - just adding some documentation before shipping :)

DarkStarSword commented 7 years ago

The current solution I'm using for admin is:

  1. Try modifying the profile with whatever privileges we have. If that succeeds awesome :) If not:
  2. Spawn rundll as admin and have it call back into us. If there are spaces in the filename (and no short filename) the DLL first makes a copy of itself in temp.
  3. Once the helper terminates (for whatever reason), reload the profiles and check if the modification was successful. If it was hooray! If it wasn't there's a new error tone to listen for: Brnk du-du-dunk (and a message in the log).

All of this happens at d3d11.dll init time, well before the DX device creation when nvapi normally loads the profile, so unless there is some weird edge case this should mean that the profile is loaded and ready to go the first time the game is run :)

I thought about popping up a dialog either before the UAC prompt, or on a failure, but given how many different ways games init themselves I felt it might be a bit risky.

This gives us a nice solution that will usually just work, popping up the UAC dialog for the user to accept (although shipping our own helper instead of using rundll would be a worthwhile change, as it would also allow us to add a meaningful description on the UAC dialog), and in the unlikely event that it doesn't, we still have the option of asking them to run as admin.

helifax commented 7 years ago

So... this option only addresses a few nvidia profile settings then? Most of them we can easily write up in a profile.... or even change with Nvidia Inspector...

If it doesn't address ALL the options a game profile can address, no offence, but this is worthless.... :( It might be viable for maybe 60% of the users without SLI or Surround or other weird setups but for the rest it means nothing... For me (in the current state) it definitely means nothing.... I will still be using my own profiles then, without a way to tell the wrapper to use this profile with these settings... :(

I find this feature incomplete in the current state :(

Sure, some of the options can be in the ini file, but without the ability to FULLY LOAD a custom profile, this is incomplete, as it creates more headache than not. For example, If I am to load my own custom profile then I need to go to the ini file and disable all this Profile stuff just for the sake of not overwriting it... This isn't normal... I'd rather have this feature missing than in the current state, to be honest...

Don't want to be negative or anything;) Just wanting to "signal" the alarm bell, that we can do much more with this! In the end, what you guys decide it will be inside;)

DarkStarSword commented 7 years ago

All options are supported, by name for those that have names, and hex ID for all. This works just fine to import your profile (only the encrypted string settings are omitted, because you really don't want that broken encoding. The duplicate setting ID will trigger an audible warning since that is an error, but it still uses the last one listed):

[Profile]
Profile "The Witcher 3"
    ShowOn GeForce
    ProfileType Application
    Executable "witcher3.exe"
    Executable "witcher3release.exe"
    Setting ID_0x00a06746 = 0x00800000
    Setting ID_0x00a06946 = 0x080020F5
    Setting ID_0x1033cec2 = 0x00000002
    Setting ID_0x1033dcd3 = 0x00000004
    Setting ID_0x106d5cff = 0x00000000
    Setting ID_0x10f9dc81 = 0x00000011
    Setting ID_0x70092d4a = 0xb19c3533 InternalSettingFlag=V0
    Setting ID_0x701eb457 = 0x2241ab21 InternalSettingFlag=V0
    Setting ID_0x702442fc = 0x1c22fe24 InternalSettingFlag=V0
    Setting ID_0x708db8c5 = 0x5c3300b3 InternalSettingFlag=V0
    Setting ID_0x70edb381 = 0x24208b6c InternalSettingFlag=V0
    Setting ID_0x70f8e408 = 0x80b671f3 InternalSettingFlag=V0
    Setting ID_0x709a1ddf = 0x4b1cd968 InternalSettingFlag=V0
    Setting ID_0x709adada = 0x37f58357 InternalSettingFlag=V0
    Setting ID_0x709adadb = 0x3F6C5C19 InternalSettingFlag=V0
    Setting ID_0x709adadb = 0x3F630FC1 UserSpecified=true
    Setting ID_0x00c96f61 = 0x0000000f InternalSettingFlag=V0
    Setting ID_0x003846a8 = 0x0000000f InternalSettingFlag=V0
EndProfile

As does this (exactly the same as above, but now decrypted, names added in comments and the duplicate setting removed - this was taken from the d3d11_log.txt after installing the above profile and massaged to keep the diff small in the git history):

[Profile]
Profile "The Witcher 3"
    ShowOn GeForce
    Executable "witcher3.exe"
    Executable "witcher3release.exe"
    Setting ID_0x00a06746 = 0x00800000
    Setting ID_0x00a06946 = 0x080020F5
    Setting ID_0x1033cec2 = 0x00000002 // NVIDIA predefined SLI mode on DirectX 10
    Setting ID_0x1033dcd3 = 0x00000004 // NVIDIA predefined number of GPUs to use on SLI rendering mode on DirectX 10
    Setting ID_0x106d5cff = 0x00000000 // Do not display this profile in the Control Panel
    Setting ID_0x10f9dc81 = 0x00000011 // Enable application for Optimus
    Setting ID_0x70092d4a = 0x00000009 // DX10DSCBNumber
    Setting ID_0x701eb457 = 0x00000001 // StereoProfile
    Setting ID_0x702442fc = 0x00004000 // StereoFlagsDX10
    Setting ID_0x708db8c5 = 0x3fa8e97b // StereoConvergence = 1.31962526
    Setting ID_0x70edb381 = 0x00000023 // StereoTextureEnable
    Setting ID_0x70f8e408 = 0x00000009 // DX10VSCBNumber
    Setting ID_0x709a1ddf = 0x00000001 // StereoCutoff
    Setting ID_0x709adada = 0x10000102 // 2DDHUDSettings
    Setting ID_0x709adadb = 0x3F630FC1 // 2DDConvergence = 0.886959136
    Setting ID_0x00c96f61 = 0x6e0326f7
    Setting ID_0x003846a8 = 0xc7729a38
EndProfile

As does this (exactly the same as above, but using setting names, floating point values and integers where appropriate, including the names for SLI settings that the driver exposes... though IMO those are a bit long winded. If you make a typo 3DMigoto will give you an audible warning to indicate that it did not recognise one of the settings, check the log for which):

[Profile]
Profile "The Witcher 3"
    ShowOn GeForce
    Executable "witcher3.exe"
    Executable "witcher3release.exe"
    Setting ID_0x00a06746 = 0x00800000
    Setting ID_0x00a06946 = 0x080020F5
    NVIDIA predefined SLI mode on DirectX 10 = 2
    NVIDIA predefined number of GPUs to use on SLI rendering mode on DirectX 10 = 4
    Do not display this profile in the Control Panel = 0
    Enable application for Optimus = 0x00000011
    DX10DSCBNumber = 0x00000009
    StereoProfile = 1
    StereoFlagsDX10 = 0x00004000
    StereoConvergence = 1.31962526
    StereoTextureEnable = 0x00000023
    DX10VSCBNumber = 0x00000009
    StereoCutoff = 1
    2DDHUDSettings = 0x10000102
    2DDConvergence = 0.886959136
    Setting ID_0x00c96f61 = 0x6e0326f7
    Setting ID_0x003846a8 = 0xc7729a38
EndProfile

As does this (exactly the same as above, but with the Geforce Profile Manager cruft removed):

[Profile]
0x00a06746 = 0x00800000
0x00a06946 = 0x080020F5
NVIDIA predefined SLI mode on DirectX 10 = 2
NVIDIA predefined number of GPUs to use on SLI rendering mode on DirectX 10 = 4
Do not display this profile in the Control Panel = 0
Enable application for Optimus = 0x00000011
DX10DSCBNumber = 0x00000009
StereoProfile = 1
StereoFlagsDX10 = 0x00004000
StereoConvergence = 1.31962526
StereoTextureEnable = 0x00000023
DX10VSCBNumber = 0x00000009
StereoCutoff = 1
2DDHUDSettings = 0x10000102
2DDConvergence = 0.886959136
0x00c96f61 = 0x6e0326f7
0x003846a8 = 0xc7729a38

As does this (not quite the same as above, but you get the idea):

[Profile]
; This setting should always be added to a profile - it is required for a
; number of other settings to work, as well as allowing the convergence to be
; saved. If you are customising a profile you should **always uncomment this**:
StereoProfile = 1
;
; This setting enables stereo compute shaders, which is requires to fix a lot
; of "one eye" type rendering issues in many DX11 games:
StereoFlagsDX10 = 0x00004000
;
; This sets the default convergence in the profile. Note that 3DMigoto will
; happily override the default value from the driver, but will only override
; the user's custom convergence if it has another reason to update the profile,
; such as a change to another setting (adding a version tag to the Comments
; setting would be one way to force an update):
StereoConvergence = 1.31962526
;
; This changes the green text that the driver displays, and is a good place to
; put any reminders to display to the user or just take some credit:
Comments = "3D Vision Fix 1.23"
;
; Change the rating: "0": 3D Vision Ready, "1": Excellent, "2": Good,
;                    "3": Fair, "4": Not Recommended
Compat = "0"
;
; If you have added some comments, you probably want to force the green text to
; show up when the game is next run. Note that like convergence, 3DMigoto will
; only override a user setting here if something else has also been updated
; (such as Comments), so this will usually only show up the first time a user
; runs the game after installing the fix (be sure to enable StereoProfile).
;StereoMemoEnabled = 1
;
; Disable compatibility mode to make sure users are seeing the real deal. Like
; convergence, 3DMigoto will respect the users custom settings here:
Disable2DD = 1
;
; Put a reminder in the green text of compatibility mode that they are not
; seeing the real deal:
2DD_Notes = "Compatibility mode is enabled. To use the fix, please disable it with Ctrl+Alt+F11"
;
; If a setting doesn't have a name or you don't know what it is (check the
; d3d11_log.txt for the names of all settings in this profile), you can use the
; hex ID (in fact, you can even paste a complete profile from Geforce Profile
; Manager - just be sure to delete any corrupt string settings if you do):
;0x1033cec2 = 0x00000002
;
; There are many more options, and we have tried to document them here -
; *please* edit this page if you figure out anything new:
; http://wiki.bo3b.net/index.php?title=Driver_Profile_Settings

0x00a06746 = 0x00800000
0x00a06946 = 0x080020F5
NVIDIA predefined SLI mode on DirectX 10 = 2
NVIDIA predefined number of GPUs to use on SLI rendering mode on DirectX 10 = 4
Do not display this profile in the Control Panel = 0
Enable application for Optimus = 0x00000011
DX10DSCBNumber = 0x00000009
StereoTextureEnable = 0x00000023
DX10VSCBNumber = 0x00000009
StereoCutoff = 1
2DDHUDSettings = 0x10000102
2DDConvergence = 0.886959136
0x00c96f61 = 0x6e0326f7
0x003846a8 = 0xc7729a38

Literally the only thing it can't do at the moment is delete a setting, but it's not clear if we need that since in most cases there is an equivalent value like 0.

DarkStarSword commented 7 years ago

Oh, and if you don't want 3DMigoto to write the profile and for whatever reason don't feel like editing the d3dx.ini, just deny the UAC prompt - it will give you an error tone but otherwise stay out of your way.

bo3b commented 7 years ago

So... this option only addresses a few nvidia profile settings then? Most of them we can easily write up in a profile.... or even change with Nvidia Inspector...

If it doesn't address ALL the options a game profile can address, no offence, but this is worthless.... :( It might be viable for maybe 60% of the users without SLI or Surround or other weird setups but for the rest it means nothing... For me (in the current state) it definitely means nothing.... I will still be using my own profiles then, without a way to tell the wrapper to use this profile with these settings... :(

I find this feature incomplete in the current state :(

Sure, some of the options can be in the ini file, but without the ability to FULLY LOAD a custom profile, this is incomplete, as it creates more headache than not. For example, If I am to load my own custom profile then I need to go to the ini file and disable all this Profile stuff just for the sake of not overwriting it... This isn't normal... I'd rather have this feature missing than in the current state, to be honest...

Don't want to be negative or anything;) Just wanting to "signal" the alarm bell, that we can do much more with this! In the end, what you guys decide it will be inside;)

Unless I'm missing something, it looks to me like this should be a superset of being able to change the entire profile.

You should be able to add a specific alternate profile like normal, for anything special like Surround or SLI that the fix doesn't already include. And, if it does already include those, it will just set the same value over again, with no impact. So, no need to edit the file to remove or change anything, unless there is a direct conflict, which I don't think happens.

Alternatively, you can make an complete profile with a different format, using this new .ini format. Should be able to make a complete profile you can just paste over the one that is there. And then not have to do the NVidia Inspector or Importer tool dance.

If this setup does not seem to meet your needs, please let us know what seems wrong and we can probably tweak it.

helifax commented 7 years ago

My bad guys;) I had the impression that only a few options would be exposed in the ini file;) from the above example. But as DarkStarSword explained, all the options are there which is awesome! The fact that 3DMigoto does some checking as well and we get the low bop is something went wrong is AWESOME!

Really want to apologise for the confusion in my head;) Really awesome! We can definitely close this one and mark as feature complete;)

Cheers!

helifax commented 7 years ago

One thing I don't quite understand;)

How to I see the value of a setting that is not encoded... For example:

Encoded: Setting ID_0x70092d4a = 0xb19c3533 InternalSettingFlag=V0

and the actual value is: Setting ID_0x70092d4a = 0x00000009 // DX10DSCBNumber

Will the log tell me the value if I write it as encoded in the ini file? 0x70092d4a = 0xb19c3533 ?

Or is there another way to extract it? Edit: Never mind found my answer: http://wiki.bo3b.net/index.php?title=Driver_Profile_Settings#Important_note_about_encoding_of_.22Internal.22_Settings

Cheers!

DarkStarSword commented 7 years ago

The d3d11_log.txt will also include decoded versions - there's a few places you can look in the log:

When it logs a current profile it does so in a format that can be copied and pasted back into the d3dx.ini if desired - I did that intentionally so I could use it to decode settings easily ;-)

And, there's an undocumented option under [Logging]: dump_all_profiles=1 will cause it to log and decrypt every profile instead of just the current one (I used that for debugging, but it could be useful in its own right).

helifax commented 7 years ago

OO, really interesting and awesome stuff there!! Big thank you for this feature;) I am beginning to get the hang of it;) I didn't know it can cope with encoded values. I had the impression I need to decode them to use them in the ini file;))

DarkStarSword commented 7 years ago

Yeah, I figured since I already needed the code to decode the values in the profile in order to reliably compare them I might as well use it to support passing in encrypted values in the format Geforce Profile Manager uses without having to worry about manually decoding them - gives us the best of all worlds :)