Steam Library no more recognized

[thebaz]

Describe the bug I am using Steam beta and since a couple of days the steam Library and its games is no more recognized in Display Magician... It is missing, I cannot even select any game.

I am adding the log files

DisplayMagician-Support-20240630-1718.zip

[Wob76]

I am having this issue too, Epic Games are showing but not steam, I am using "Steam Families Beta" and it was working fine a couple of weeks ago when I last used a game.

Steam Beta Branch: Steam Families Beta Steam Version: 1719618357 Steam Client Build Date: Sat, Jun 29 10:26 AM UTC -08:00 Steam Web Build Date: Sat, Jun 29 9:20 AM UTC -08:00 Steam API Version: SteamClient021

DisplayMagician 2.5.0.352

[terrymacdonald]

Gah. Sounds like they've updated the steam file formats :(. So annoying when they do that as the whole Steam ecosystem needs to reverse engineer the new changes and then build them into our libraries. Le sigh.

It will be a few weeks before we can get this fixed.

Thanks

Terry

Message ID: @.***

com>

[Wob76]

Painful, thanks for all your efforts.

[thebaz]

Thanks Terry!

[terrymacdonald]

@thebaz Which Steam Beta are you on? Steam Beta Update, or the Steam Families Beta? I need to know if I have to test both or just one of the two options.

Thanks Terry

[thebaz]

Steam Beta Update... I also tried to revert to normal version without any luck or change

[terrymacdonald]

Thanks @thebaz, that helps.

So after some research Steam have indeed updated to a new Appinfo.vdf file format. This adds a new type of binary object (StringPool) that requires an update to the underlying ValveKeyValue library in order to support it. This is still under development by the amazing ValveKeyValue team here: https://github.com/ValveResourceFormat/ValveKeyValue/pull/99.

I've asked them for some more information on how long it will be before they can release a binary version to Nuget so that I can incorporate it into a new DisplayMagician release. I expect it will be a week or two before it's released, and then I will make a new hotfix version and release it as DisplayMagician 2.5.1.

This is the git branch you can follow along if you want to see where we're up to: https://github.com/terrymacdonald/DisplayMagician/tree/hotfix-steam-beta

TLDR - I can't really do anything more until the upstream library is updated, and some very smart people are working on it now, so hopefully we have a solution soon.

Thanks Terry

[thebaz]

Thank you Terry for the update

[thebaz]

I am trying to find a workaround to this problem but I cannot find a solution... So I did something like this:

where "Executable to start" is: D:\SteamLibrary\steamapps\common\Assetto Corsa Competizione\acc.exe

and "Wait until an alternative executable is closed before continuing" is: D:\SteamLibrary\steamapps\common\Assetto Corsa Competizione\AC2\Binaries\Win64\AC2-Win64-Shipping.exe

But the programs started in step 3 are closed after the game is started

What am I doing wrong?

[terrymacdonald]

That should be working. You'll have to create a support zip file for me to see what's going on.

Please go into Settings and change the logging level to TRACE, then try to run that game shortcut. Once it's failed to work, hi back into Settings, then click on the Create Support ZIP File' button, and save the zip file. Come back to this issue in your web browser, and then upload the zip file.

I'll have a look at it to see if I can see what's wrong.

Thanks Terry

On Mon, 1 Jul 2024, 10:16 pm Luca Bassini, @.***> wrote:

I am trying to find a workaround to this problem but I cannot find a solution... So I did something like this:

image.png (view on web) https://github.com/terrymacdonald/DisplayMagician/assets/7090323/44d71b4a-71af-4f94-98b2-c31835d2025c

where "Executable to start" is: D:\SteamLibrary\steamapps\common\Assetto Corsa Competizione\acc.exe

and "Wait until an alternative executable is closed before continuing" is: D:\SteamLibrary\steamapps\common\Assetto Corsa Competizione\AC2\Binaries\Win64\AC2-Win64-Shipping.exe

But the programs start in step 3 are closed after the game is started

What am I doing wrong?

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2199763222, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LAMKVWTZD7SJ4EBFGTZKET6RAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJZG43DGMRSGI . You are receiving this because you were assigned.Message ID: @.***>

[thebaz]

Here is the zip file DisplayMagician-Support-20240701-2327.zip

[terrymacdonald]

@thebaz Was that a brand new gameshortcut that you did for ACC? Or was it your old ACC shortcut that you changed to run as an Executable?

If it was a changed shortcut, I'm wondering if there was some old settings that are still in there that are causing issues. Can you try a brand new shortcut and try just running ACC without any start programs to see if that works?

Thanks Terry

[terrymacdonald]

BTW - The team at ValveKeyValue are working on the issue: https://github.com/ValveResourceFormat/ValveKeyValue/pull/100

Steam have added a dynamic stringtable as a new data type, so the team are trying to figure out how best to handle that. Could be a couple of weeks until they have a solution that works, and until they have pushed a new version of the library we need.

[thebaz]

It was the old shortcut, but with a new one the problem persists DisplayMagician-Support-20240703-1150.zip

[terrymacdonald]

@thebaz I have no idea why this isn't working, as my version of ACC works fine when I run it in my local copy of DIsplayMagician.

I have just committed a Steam fix that uses the latest ValveKeyValue library that the team have pushed out recently, but it ONLY will work with .Net Standard....meaning that I cannot fix the DisplayMagician 2.5.x branch (main), and instead can only fix the DisplayMagician 2.6.0 branch (develop).

This means adds a massive complication, as I was planning on updating the main NVIDIALibrary code to completely change over to some code taken from Soroush Falahati's NvAPIWrapper.net project, but that code is nowhere near ready for primetime release. This contrasts with the need for a fast release to support the latest Steam beta....

The only solution that I can see is to release an interim DM v2.6.0 shortly with all the fixes so far (new windows installer, .net 8.0, latest libraries, numerous fixes), and then to hold the NVIDIALibrary changes to a later DM v2.7.0. This means I will need to create a release branch where I tweak the code for a v2.6.0 release. This will take a few days.

If I do the v2.6.0 interim release, I will need people to beta test the version until we iron out all of the bugs. I'm hoping to get a beta test version out of v2.6.0 within the next week.

Are you able to test v2.6.0-bet when I release it?

Thanks Terry

[Wob76]

@terrymacdonald I can put my hand up for testing if you need.

[thebaz]

@terrymacdonald Yes, I'll be able to test the beta version when you eventually release it

[terrymacdonald]

Hi All,

Time for an update.

The fix is in the code and works (also we now support 3rd apps and games added to Steam library too!) but my current hold up is actually in the packaging of the application into a windows installer. I've been testing different ways to do this as the move from .net framework 4.8.1 to .net core 8.0.0 has been a massive change. It has resulted in a completely different set of binaries and dlls and other files being created, as well as a different build process too. This is all taking some time to figure out.

I've gone from a WIXv3 based installer to a windows default installer to an NSIS installer then back to a WIXv4 installer all within the last month. It's necessitated me building a brand new way of handling the desktop background context menu, and even down to which libraries are used within the application. It's been a ton of work.

I'm now down to the last few bugs to iron out, and to finish the new packaging process. I think I should have something ready for everyone to test within the next month (assuming there aren't anymore show stoppers.

My goal is to then send out messages through this issues list to get you all to test some DM v2.6.0 test builds, and then once the major bugs are ironed out I will push out the DM v2.6.0 beta to those users who opted in the beta builds. And then I'll finally push the DM v2.6.0 update to the main user group.

Lastly, I've decided to hold back the NVIDIALibrary update for now until DM v2.7.0 as it requires a LOT more work testing and troubleshooting many errors. The Steam beta client changing file structures has forced my hand here a little bit, so a better NVIDIALibrary will just have to wait for now.

Thanks Terry

[thebaz]

Is there any news on this topic?

[terrymacdonald]

Hi @thebaz ,

Yeah I've been slowly working through the bugs in the latest version, trying to get them squashed for a release. I've had to change the JSON file format for how DM stores it's settings, which also means I've had to build an upgrade process for patching the existing settings so that they will still work in DM 2.6.0 when released. This in turn meant that I had to change how the settings were processed, which in turn means even more code changes.

This version of DisplayMagician will probably have a lot of bugs when we do finally get to testing it. There are a lot of major, core changes to how it works, and that means there are highly likely to be bugs present. We'll have to do a lot of testing to make sure we catch the major ones before release.

I'm currently working through upgrade testing at the moment, trying to test all the different ways that DM can be run and trying to make sure that the upgrades work without any errors. There are also some slight UI tweaks I need to make then I think it will be ready for initial alpha testing to people via these github issues.

I'm currently really busy in $dayjob, which means I don't get much free time to do programming, so the development time has slowed down more than I'd like. Not much that I can do about that, but I wanted to set some realistic expectations about when this will be released. It could still 3-4 weeks before I'm comfortable that it's reliable enough for an alpha release.

TLDR; It's getting closer, but there is still a bit to do before it's ready for a full release.

Thanks Terry

[terrymacdonald]

I forgot to mention that the new WIXv4 installer bundle seems to be working, though there is still an issue where the taskbar icon doesn't show the proper DisplayMagician icon. I've still got some testing to do there too! But at least the installer will install .net 8.0 if its not already installed on the target computer, which means people don't need to download .net 8.0 themselves in order to get DM to work.

[thebaz]

This version of DisplayMagician will probably have a lot of bugs when we do finally get to testing it. There are a lot of major, core changes to how it works, and that means there are highly likely to be bugs present. We'll have to do a lot of testing to make sure we catch the major ones before release.

Hello @terrymacdonald,

it won't be a problem if a new version will probably have a lot of bugs... Actually I can't have my Steam Library recognized and I would be glad to test and help solving the problem

[herve4545]

I am OK for testing V2.60 I am on beta DisplayMagicianSetup-v2.6.0.524

[terrymacdonald]

Hi @herve4545,

One thing to keep in mind is that the v2.6.0 that you've been using is actually quite a bit different to the version that will be sent out for testing. I've had to convert DisplayMagician from a .net framework 4.8 application to a .net 8.0 application (which is a huge backend change) which then also required a new way of installing it.

That then required new libraries, some of which didn't support .net 8.0, which meant I had to find alternatives or rebuild the libraries in .net 8.0 myself. Then of course there were many bugs that required changing.

What this means is that the early versions of v2.6.0 aren't the same as v2.6.0 we're testing now.

In fact, I think it might be best that I actually change the versioning to v2.7.0 to highlight how different this version actually is. I'll have a think about that over the next little while.

Thanks Terry

On Sat, 14 Sept 2024, 3:47 am herve4545, @.***> wrote:

I am OK for testing V2.60 I am ib beta DisplayMagicianSetup-v2.6.0.524

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2349260565, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LEBELN36EA762VVU73ZWMCJHAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBZGI3DANJWGU . You are receiving this because you were mentioned.Message ID: @.***>

[terrymacdonald]

Hi!

This may be fixed by the brand new version of DisplayMagician I just released. I would suggest that you try to the new DisplayMagician v2.7.0-alpha.1 test build available here: https://github.com/terrymacdonald/DisplayMagician/releases/tag/v2.7.0-alpha.1

It should fix your problem, but it may also cause other problems instead. This version has had a complete backend change, new installer, new features, and so it may have some bugs that need squashing.

Please test it out. and if you find any new problems please log a new issue. If it fixes this issue then please update this issue to tell me and then close this issue (it helps me keep track of what I've still got to fix).

Thanks Terry

[thebaz]

Thanks @terrymacdonald

Can't wait to go back home and try!

[thebaz]

Hello @terrymacdonald

Thank you very much for your effort on this great utility! Everything works as expected but I have a question. Is the DisplayMagicianConsole.exe actually missing?

[terrymacdonald]

It was there when I last tested it? I'll look again tonight on my test machine.

Thanks Terry

On Fri, 4 Oct 2024, 5:52 am Luca Bassini, @.***> wrote:

Hello @terrymacdonald https://github.com/terrymacdonald

Thank you very much for your effort on this great utility! Everything works as expected but I have a question. Is the DisplayMagicianConsole.exe actually missing?

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2391883744, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LDYHGDURJ26GAX4MYTZZVY4VAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJRHA4DGNZUGQ . You are receiving this because you were mentioned.Message ID: @.***>

[Manfredk2]

the v2.7.0-alpha.1 seems to work as expected. Thank you very much, Terry.

I am having a small problem but i dont remember if this was already in the 2.5 stable version.

I am running Raceroom from steam. Raceroom has some starting options like "play raceroom with dxvk". When i start raceroom over my new generated game shortcut it starts Raceroom in DX9 mode instead in Vulkan/DXVK mode. Starting Raceroom in steam launches it in DXVK as it is set as default starting option.

[terrymacdonald]

@thebaz Darn it! Looks like the DisplayMagician packaging code only took the basic DisplayMagicianConsole.exe, rather than the 'Single Executable' version of DisplayMagicianConsole.exe.

As DisplayMagician is now a .net 8.0 application, it works in a much different way. The way it is built is now a 2-step process - the app is built, and then packaged. The packaging I've chosen is to build a single exe app. What this means is that DisplayMagicianConsole.exe should be a single file of about 25MB.

The problem that happened is that we are currently grabbing the built DisplayMagicianConsole.exe (about 256KB) instead of grabbing the published DisplayMagicianConsole.exe (about 25MB). This is the reason you're getting an error when running it.

It's a little tricky for me to get the publish build to work thanks to the way that Microsoft have decided to handle publishing via VisualStudio, and I'm right in the middle of changing how version information is shared amongst DM components, but I do now know what I need to fix.

Give me a few days and I should have another alpha build for you to test.

Thanks Terry

[terrymacdonald]

Hmmm. For that mode are there any command line options you can add to the DisplayMagician shortcut? You may need to pass them through the exe.

If that doesn't work please log a new issue as that is a whole other thing I'll need to troubleshoot.

Thanks

Terry

On Sat, 5 Oct 2024 at 20:40, Manfredk2 @.***> wrote:

the v2.7.0-alpha.1 seems to work as expected. Thank you very much, Terry.

I am having a small problem but i dont remember if this was already in the 2.5 stable version.

I am running Raceroom from steam. Raceroom has some starting options like "play raceroom with dxvk". When i start raceroom over my new generated game shortcut it starts Raceroom in DX9 mode instead in Vulkan/DXVK mode. Starting Raceroom in steam launches it in DXVK as it is set as default starting option.

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2394967252, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LGJ4J2YVTHRURSMIQLZZ6JVDAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJUHE3DOMRVGI . You are receiving this because you were mentioned.Message ID: @.***>

[Manfredk2]

I asked for possible command line options at the Raceroom forum. Lets see if there is an answer. Tried already "-vulkan" or "-dxvk" without success.

CrewChief also has an option to start Raceroom automatically. But here it starts also in DX9

[Manfredk2]

copying the dxvk dll´s from the dxvk github to the DX9 folder of Raceroom does the trick. Now Raceroom starts with Vulkan instead of DX9

[terrymacdonald]

Ah awesome news. So is it a Raceroom issue or a DisplayMagician one do you think?

Terry

On Sun, 6 Oct 2024, 7:03 am Manfredk2, @.***> wrote:

copying the dxvk dll´s from the dxvk github to the DX9 folder of Raceroom does the trick. Now Raceroom starts with Vulkan instead of DX9

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2395135074, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LAJ573JGTJ35RZZZADZ2ASWZAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVGEZTKMBXGQ . You are receiving this because you were mentioned.Message ID: @.***>

[Manfredk2]

i am no expert but i would say it is a Raceroom thing. There is a folder inside Raceroom called x64dxvk. I was told to start the Raceroom .exe inside this folder to get Vulkan, but this did also not work. It worked only when starting direct from Steam. So for me it is no DM issue.

[terrymacdonald]

Hmmmm. It could be something to do with the working directory not being set by DisplayMagician. I'll have a look at the code tonight and see if there is something missing there when it's being run as a stream library item.

Thanks Terry

On Sun, 6 Oct 2024, 7:32 pm Manfredk2, @.***> wrote:

i am no expert but i would say it is a Raceroom thing. There is a folder inside Raceroom called x64dxvk. I was told to start the Raceroom .exe inside this folder to get Vulkan, but this did also not work. It worked only when starting direct from Steam. So for me it is no DM issue.

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2395314488, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LD6MZRTODYCFSL5S7TZ2DKOZAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVGMYTINBYHA . You are receiving this because you were mentioned.Message ID: @.***>

[Manfredk2]

This could be. Maybe the RRRE.exe from x64dxvk-folder is looking for d3d9.dll and dxgi.dll in the wrong folder. Both dll are in the x64dxvk-folder but not in the x64-folder.

[terrymacdonald]

I might try an experiment with Raceroom and see if I can find anything odd.

On Mon, 7 Oct 2024, 7:07 pm Manfredk2, @.***> wrote:

This could be. Maybe the RRRE.exe from x64dxvk-folder is looking for d3d9.dll and dxgi.dll in the wrong folder. Both dll are in the x64dxvk-folder but not in the x64-folder.

— Reply to this email directly, view it on GitHub https://github.com/terrymacdonald/DisplayMagician/issues/317#issuecomment-2395983588, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLU5LA5464R36QBZGWDAQ3Z2IQLFAVCNFSM6AAAAABKEGDCGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVHE4DGNJYHA . You are receiving this because you were mentioned.Message ID: @.***>

[terrymacdonald]

I just checked the working directory to see if its set, and it is always set when a process is started in the code, so its unlikely to be an issue with the way that DM starts the process....

ProcessStartInfo psi;
if (File.Exists(executable) && IsExecutableFileType(executable))
{
    // Is exe file 
    if (runAsAdministrator)
    {
        psi = new ProcessStartInfo(executable, arguments)
        {
            UseShellExecute = true,
            Verb = "Runas",
            CreateNoWindow = false,
            RedirectStandardOutput = false,
            WorkingDirectory = Path.GetDirectoryName(executable)
        };
    }
    else
    {
        psi = new ProcessStartInfo(executable, arguments)
        {
            UseShellExecute = false,
            CreateNoWindow = false,
            RedirectStandardOutput = false,
            WorkingDirectory = Path.GetDirectoryName(executable)
        };
    }

}
else
{
    // Isn't a file (somethign like a url), or is a file but isn't an executable
    psi = new ProcessStartInfo(executable, arguments)
    {
        UseShellExecute = true,
        Verb = "Open",
        CreateNoWindow = false,
        RedirectStandardOutput = false,
        WorkingDirectory = Path.GetDirectoryName(executable)
    };
}

[terrymacdonald]

ok I believe this is fixed now in the latest alpha, so I will go ahead and close this issue for now. Please feel free to reopen it if you still have the issue after testing it.

Thanks Terry