search

sonic2kk / steamtinkerlaunch

Linux wrapper tool for use with the Steam client for custom launch options and 3rd party programs
GNU General Public License v3.0
2.17k stars 73 forks source link

⚠ MO2 - Mod Organizer 2 overhaul discussion #598

Closed frostworx closed 1 year ago

frostworx commented 2 years ago

This issue is meant to be a generic brainstorming issue for a possible MO2 overhaul. Feel free to contribute any ideas you have to improve it. Of course PRs are very appreciated as well as usual.

Starting with this proposal: If dotnet is no longer required at all for MO2 (can someone confirm it is no longer required for modding?), it might make sense to simplify the whole implementation by using one global "install directory" (=MO2 Program Files dir) and f.e. symlink it into the corresponding compatdata without any further installation. Imho this would help massively to solve almost all problems people have with the confusing "gamemode" and "standalone modes".

sonic2kk commented 2 years ago

DotNet was removed a while ago, right? I modded Oblivion and New Vegas around June-July time and didn't really have any problems (using Proton 7.0, forget what I used for Oblivion but I used 7.0-4 for New Vegas). So in my own experience, DotNet shouldn't be required anymore. Other users have reported modding Oblivion with success too on the issue discussing the recent MO2 download patch.

Maybe any work and testing for this could be done on a separate branch? That can get... Tedious, but to preserve existing mod installs and get feedback having a separate branch once any kind of work starts on this might be nice.

Just my thoughts though, I've only modded 3 games that use similar engines and modding processes. It's very possible there are problems in other areas I haven't encountered. Though I will say the recent link to the Rockerbacon(?) ModOrganizer 2 project mentions Wine having "built-in support" which I assume means that MO2 should be fully compatible with Wine. But this project seems to mainly focus on Bethesda games, which is the extent of my experience too.

I would guess that MO2 might not require DotNet but some of the other GUI applications that you can launch via MO2 for the likes of Skyrim character editing might need DotNet. I couldn't say for sure though.

Just realised the tone of my reply might not get it across (sorry!) but I think being able to clear up any confusion here is a great idea :)

frostworx commented 2 years ago

yeah we recently discussed this already, but I haven't found the corresponding issue anymore to refer it here (haven't searched for long and assumed you'd reply here anyway again. thanks!) You would not beliebe how much time I spent with testing and writing all that MO2 stuff (without actually using it...). It once worked fine but apparently it doesn't anymore... It would be really "nice" if people interested in MO2 would take part in a discussion here and contribute what they know. Although it seems likely that dotnet is no longer required anymore, I would appreciate it if several people could confirm this at least.

frostworx commented 2 years ago

Of course when removing the Standalone Mode, it is no longer possible to see all modable found games in MO2 and switch between them, because the compatdata of one game doesn't have "the other games installed" (it should work with doing the same symlink voodoo which is currently used in the Standalone Mode compatdata, but messing around with symlinks in the game compatdata should be avoided obviously and therefore isn't an option)

So if you like the existing option to have all modable games in one mo2 running and would like to keep that function(=Standalone Mode) leave a comment.

If you don't care about this and would prefer one MO2 per game (in its own compatdata, so a rewritten game mode and much more simplified as the current implementation - so pretty much without any possible confusing trouble) leave a comment as well.

sonic2kk commented 2 years ago

So if you like the existing option to have all modable games in one mo2 running and would like to keep that function

Oh! I do use this every so often and its quite useful. Having a global MO2 install works basically like it does on Windows. Just voicing my support for keeping it :bow:

frostworx commented 2 years ago

Just tried it and agreed, it still seems to work fine. As this would have been the main task to rewrite the mo2 implementation, I might just cancel the idea to rewrite it generally. Haven't checked yet why the nxm urls do no longer open, but as the code didn't change here and the desktop files are still identical to those created months ago, I'd think it might be possible, that this is some "external" problem.

After installing mo2 during above test (started with steamtinkerlaunch and mo2 configs), I recognized that the "Connect to Nexus" button under mo2->settings->nexus doesn't do anything. I am not sure if this is related, but it could also explain the missing nxm url functionality (as in communication between wine/proton to the browser does not work at all. If so, there's nothing to fix on steamtinkerlaunch side)

sonic2kk commented 2 years ago

I'll have to retest but recently when I was testing this for another issue, I was able to use the "Connect to Nexus" button. It opened my default browser and prompted me to authorise. This happened with both Standalone Mode and Game Mode.

frostworx commented 2 years ago

snippet from WINEDEBUG:

00c4:fixme:system:DisplayConfigGetDeviceInfo Unimplemented packet type: 11
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (000000002D008DC8, 30005, 0000000000115B20 {VT_EMPTY}, 0000000000115B00 {VT_BSTR: L"Connect to Nexus"}): stub
sonic2kk commented 2 years ago

I forget which version of Proton I was using, I think I was just using 7.0-4 in Game Mode. I do my testing for this on my laptop because it's separate from my modded installs on my PC, so when I get a chance to test on my laptop again I'll see what's up on my side :smiley:

frostworx commented 2 years ago

those lines are written when "Connect to Nexus" is clicked: (with WINEDEBUG enabled):

00c4:fixme:win:GetPointerDevices (00000000001162E0 0000000000000000): partial stub
00c4:fixme:system:DisplayConfigGetDeviceInfo Unimplemented packet type: 11
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (000000002AF15B28, 30005, 0000000000115AF0 {VT_EMPTY}, 0000000000115AD0 {VT_BSTR: L"Cancel"}): stub
00c4:fixme:bcrypt:BCryptGenRandom ignoring selected algorithm
00c4:fixme:uiautomation:UiaRaiseAutomationPropertyChangedEvent (0000000028402E88, 30047, 00000000001162D0 {VT_EMPTY}, 00000000001162B0 {VT_R8: 317.000000}): stub

The mo2 in-gui log also logs an "opening url" line, so very likely this is the core issue (and therefore not steamtinkerlaunch related)

the mo2 internal logfile

~/.config/steamtinkerlaunch/mo2/compatdata/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer/nxmhandler.log: also logs:

[So Sep 25 18:35:38 2022] EnableNonClientDpiScaling() failed for HWND 0x300d6 (120) (Aufruf nicht implementiert.)
[So Sep 25 18:35:38 2022] Unable to enumerate family ' "Noto Sans Inscriptional Parthian" '
[So Sep 25 18:35:41 2022] "C:/Modding/MO2/nxmhandler.exe" "reg" "cyberpunk2077" "C:/Modding/MO2/ModOrganizer.exe"

Aufruf nicht implementiert means function not implemented, which probably is the same issue.

frostworx commented 2 years ago

played a bit around: I found the problem why the nxm protocol download no longer works. should be easy to fix.

damn, I didn't.

edit: I actually tested the nxm download handler and it just works for me multiple times reproducible...!?

edit2: not 100% yet, but seems like something in ModOrganizer.ini enables/borks the functionality. no time left for today, maybe more tests tomorrow...

edit3: nope, mo2 seems to ignore mo2 sometimes when it is already open, but seems like it is always reproducible that it opens it when closed and starts reliably the download then.

not testing any further for now - works for me and possibly not steamtinkerlaunch related.

frostworx commented 2 years ago

Yay, impressive feedback overall so far! 👍

I wonder what it takes to simply keep the "global instance" implementation for the "Standalone Mode", and implement a "portable instance" per game for the "Game mode" and sync (symlink?) all required files (e.g. downloads) to avoid duplicates. Of course the "Modes" would be renamed accordingly to match the upstream global and portable instances. This would help of course to also add wabbajack support (if feasable) Undecided yet, if I'll look into this... I'll dive a bit into a basic approach to see how much work this will be. wip (don't hold your breath)

Current guess: shouldn't be too complicated, but relatively big

edit: I was on a short trip for the last few days and lost the current wip state. no idea when I can pick it up again for now.

sonic2kk commented 2 years ago

Haven't had time to look into the code yet but I just saw on the MO2 wiki that in the desktop file template I added, the STL path in the Exec line was /usr/bin/SteamTinkerLaunch blah blah.

If STL hardcodes its own path in the MO2 desktop file that's something that could be changed. STL on Deck (and with ProtonUp-Qt) is at ~/stl/prefix for example.

Since we also can't always rely on STL being on the path (manual non-root installs where the script could be anywhere), is there a way we can set this to the STL script?

Sorry if this is already being done, it's very possible the template I copied set that path because that was my installation path to STL. But I wanted to flag this up just to make 100% sure 😃

Retrograde-i486 commented 2 years ago

So I'm not sure how helpful I can be but I just wanted to say that I just completed a fully modded run of Fallout 4 with STL and Mod Organizer 2 and it worked perfectly. I've been keeping my system up-to-date (Garuda Linux 5.19.11-zen1-1-zen x86_64) and I didn't think I was updating STL but upon checking I guess it's been getting updated since I'm on v11.11 which appears to be the latest release.

From what I've been seeing it seems people are having the most issues with the flatpack version?

frostworx commented 2 years ago

Thank you very much for confirming @Retrograde-i486! It is even very helpful, because usually people only say something when something does not work :) (I'm still working on a rewrite to simplify MO2, but it will need some more time until done)

Retrograde-i486 commented 2 years ago

OK good to hear, I've been running this install of MO2 and STL since July and every time I would update my system I'd check and see if the game still loaded up with the notion that I'd restore from a snapshot if it didn't. Through every Wine and STL update I kept waiting for something to break and it never did so I'm happy to report that I completed all the DLCs and the main quest modded on Linux:

Fallout 4 Boom: https://imgur.com/01tRe1e

MO2 List: https://imgur.com/4MP7n5u

If anyone has any questions I can try and help

frostworx commented 2 years ago

Wow, thank you very much! Highly appreciated!

KBouder commented 2 years ago

Quick question about MO2 under proton, is it supposed to be a extremely slow and unresponsive? Is there any way to improve performance while using it?

frostworx commented 2 years ago

no, it works normally. not steamtinkerlaunch or topic related though

KBouder commented 2 years ago

no, it works normally. not steamtinkerlaunch or topic related though Sorry, for being off topic. It definitely does not run normally for me for some reason.

sonic2kk commented 2 years ago

If you're using a version of GE-Proton the issue might come from its theme, you can disable by enabling Winecfg -> Play -> Desktop Integration -> Theme -> Choose "(No Theme)"

Apart from that I'm not sure why it would run slowly. Might be worth opening a separate issue with more details for this if it persists and if you have the time :-)

frostworx commented 2 years ago

after some funny issues comments I started spending at least some minutes on the MO2 case before the weekend (yay, finally)

short description:

steamtinkerlaunch generates a small mo2 download script (MODLER in the code if someone is curious) in /dev/shm/steamtinkerlaunch/ which is triggered via steamtinkerlaunch mo2 u (which in turn is triggered by the nxm mimetype/desktopfile).

When clicking a nxm url, the script is started with the mod url being appended as command line argument though the steamtinkerlaunch function dlMod2nexurl. The script contains game instance specific entries and therefore can't be used as generic script. For that reason, the script is auto-generated on demand and removed when steamtinkerlaunch closes. I think (not yet tested) that the download only works fine until that script is still available - means the "else part" (the script MODLER is not available) needs to be debugged, tested and possibly replaced. Just a wild guess so far, but it would explain certain reports/experiences.

edit: of course I will check that and add it to the list of the to-be-rewritten/tested mo2 stuff.

edit2: I think it could be possible, that the download script could be generic (and therefore does not be created) when using a portable instance. I'll elaborate on a separate comment what I'm trying to achieve with the rewrite (hopefully understandable 😏)

frostworx commented 2 years ago

summary of the current rewrite plan:

KBouder commented 2 years ago

If you're using a version of GE-Proton the issue might come from its theme, you can disable by enabling Winecfg -> Play -> Desktop Integration -> Theme -> Choose "(No Theme)"

Apart from that I'm not sure why it would run slowly. Might be worth opening a separate issue with more details for this if it persists and if you have the time :-)

I'll give that a shot and see what happens (pretty sure I'm using GE35 currently. I won't have time to troubleshoot this until Monday or Tuesday. Currently installing or enabling any mod takes like 1-2 minutes for each step (regardless of mod size) and the application hangs during that time. Even mod installation windows have this issue especially when there are a lot of check boxes, some will even crash MO2 entirely.

sonic2kk commented 2 years ago

Hmm, so is MO2 unresponsive for you if you have a lot of mods installed? Or more accurately, a lot of large mods? Or is it just laggy/unresponsive in general?

KBouder commented 2 years ago

Hmm, so is MO2 unresponsive for you if you have a lot of mods installed? Or more accurately, a lot of large mods? Or is it just laggy/unresponsive in general?

Laggy even with no mods. I don't even want to try and load my actual modlist, I don't see that working well at all.

frostworx commented 2 years ago

please be so kind and open a separate issue for this

frostworx commented 2 years ago
  • If someone has hints what is required for sharing the list of enabled mods with the game instance of the "global" setup with the "portable" setup any hints welcome (for example merging ModOrganizer.ini parts already is enough?)

I think collaboration is required here to proceed correctly from scratch.

1) It makes sense anyway, but it looks like we even have to share the download directory because the ".meta" files besides the downloaded files also contain the information if a mod is installed.

2) other files which need to be synced/mirrored

3) more?

frostworx commented 2 years ago

nxm protocol plan:

edit: excellent timing with playing with a new download implementation: the api for all mod tools is 503 (service unavailable) at the moment... the api is back, but enough for today anyway :)

sonic2kk commented 2 years ago

Oh boy okay, so I spent the afternoon going through this in as much detail as I could to try and share what I know and what I was able to find out. I really hope some of this can be helpful but man this got really lengthy, sorry for such a long read.

Is there an official configuration option to define a download directory path? (ModOrganizer.ini doesn't seem to have an option)

Yes, however these are not created until the user modifies the path themselves. I'll give a bit of background first:

The download directory is per-game and in the Wine prefix by default. MO2 by default has a "base directory", this is the path to the ModOrganizer 2 instance for your game. This base directory is used to create the paths to other related instance folders. Maybe this screenshot from the MO2 settings will help explain it.

image

By default the "actual" path for the base directory on your system is $HOME/.config/steamtinkerlaunch/mo2/compatdata/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer/<GameName>. This ModOrganizer folder has a list of game instance folder names, which is where actual mods seem to be installed, any mod overwrite files, and of course the mods downloaded from Nexus.

This means changing Base Directory would change the download folder. I haven't tried to change this folder so I'm not sure what would happen if you tried to point it to somewhere outside of the base directory (no matter where that is). I would assume it would work though.

Note: These paths seem to be per instance and not per profile, so you can't have one profile for an instance download mods to another folder than another profile. All profiles for a game instance share the same directories.

Okay, so with that background out of the way: The ModOrganizer.ini file will contain the paths for Downloads, Mods, Caches, Profiles, and Overwrite (as listed in the screenshot above) but only when a user manually changes these paths. I manually changed these paths and opened the ModOrganizer.ini file to see their values. Here is a snippet:

[Settings]
download_directory=%BASE_DIR%/downloads/TestDownloads
cache_directory=%BASE_DIR%/webcache/TestCaches
mod_directory=%BASE_DIR%/mods/TestMods
overwrite_directory=%BASE_DIR%/overwrite/TestOverwrites
profiles_directory=%BASE_DIR%/profiles/TestProfiles

These were under the [Settings] block in the ini file, and they were actually at the bottom of that block, but I don't think that matters too much. If you change these folder names back to their "default", which are %BASE_DIR and the folder name shown in the above screenshot (for mods the default path is %BASE_DIR%/mods for example), then this line will be removed from the ini file. In short, the paths only show up in the ini file if they're different from the defaults that MO2 probably has hardcoded checks to look for - even if the %BASE_DIR% is different, if the path is still%BASE_DIR%/modsor whatever, the associatedblah_directory=line in theModOrganizer.inifile will be removed. So if the base directory is different but the relative paths to those folders hasn't changed from the default, MO2 will not store these in theModOrganizer.ini` file.

The %BASE_DIR% is not actually defined anywhere in the ini file by default that I can see. I guess it is always just hardcoded to C:\users\steamuser\AppData\Local\ModOrganizer\<GameName>. If this was changed it would probably show up in the ini file. I can't change the base directory at the moment (would test on my laptop but I can't at the moment, doing this testing with my for-real modded installs and living on the edge!) but it would probably be named something like base_directory= and put under the [Settings] block though I'm not sure if it would have quotes or anything, or have C:\ or C:/, probably forwardslash as that would mirror what is in the actual path in the MO2 UI. Since %BASE_DIR% just mirrors exactly what is in the MO2 UI I would hazard a guess that MO2 will just write out what is in the Base Directory textbox in the MO2 UI, and MO2 is smart enough to parse that when it's opened. This part might need more testing though.

Now, something important I want to mention is, when I tried to change the mods path in the MO2 settings, it gave me this warning:

image

This is basically saying that if there are different or no mods in the new mods folder, MO2 can't load the mods. But the error was a lot less scarier than I thought: Changing the "Mods" directory back to what it was before by just editing the line in the textbox from the MO2 settings UI worked no problem - This change is very easy to revert and the mods all work once you change it back to a folder with mods that MO2 can recognise. It really seems like no big deal, but backing up profiles is probably the way to go since if you change other paths or install different mods it might confuse MO2 (and profiles contain information about what mods to load and the order to load them in - Which I go over a bit later on).

There are also a couple of potentially relevant symlinks: The game's Proton prefix has a symlink to the AppData/Local/ModOrganizer directory (/path/to/steamapps/compatdata/<appid>/pfx/drive_c/users/steamuser/AppData/Local/ModOrganizer) This points to the same ModOrganizer base directory folder has described above, under the STL-created MO2 compatdata.

image

Something minor note: Running LOOT (which is a load-order sorting program that comes built into MO2) creates a folder at /path/to/steamapps/compatdata/<appid>/pfx/drive_c/pfx/users/steamuser/AppData/Local/LOOT/<Game Name> with a file called masterlist.yaml. This seems to just be a LOOT-specific file that manages what outputs it should give depending on what mods or potential mod conflicts it finds. It seems like it also pulls stuff down from git as this LOOT folder also has a .git folder.

For game Proton prefixes, this is created once when LOOT is ran. So for example the New Vegas Proton prefix will only have a FalloutNV folder, and Oblivion will only have an Oblivion folder.

However, this structure is also created at the STL-created MO2 prefix - $HOME/.config/steamtinkerlaunch/mo2/compatdata/pfx/drive_c/users/steamuser/AppData/Local/LOOT - And each game you run LOOT with in MO2 Standalone Mode will create game folder here. Here's a picture to illustrate better, you can see I have a FalloutNV and Oblivion folder in the MO2 prefix.

image

This totals about 11mb. Not a big deal at all and should not impact functionality, but I wanted to mention this as its related to files MO2 can create (by running LOOT, which comes preinstalled with MO2) and is a side effect of running in Game Mode and Standalone Mode.

how to proceed if the download dir in the game compatdata already contains files? Any thought?

So if it has actual mod files for the game, it could end up being a potential problem. the .meta file for the downloaded mods contains information about the mod and whether or not it is installed. I did a couple of tests:

This seems to be MO2 standard behaviour, I'm not sure if it's something STL necessarily has to worry about. Like with MO2 in general I think we should expect users to know how to manage their mod configuration and so we should expect them to make sensible choices around where they download mods and set mods to be downloaded to.

profiles/Default/modlist.txt seems to list all used mods per game (?)

Sort of. The way MO2 works is, there is usually a Default profile. "Profiles" in MO2 allow you to setup separate, independent mod lists. So if you wanted to play New Vegas with a bunch of vanilla game tweak mods you could create a profile called "VanillaTweaks". Then if you wanted to install some major gameplay changing mods, you could duplicate that VanillaTweaks profile and install new mods to a modlist that are completely separate and independent.

This allows you to switch between different mod profiles per game. It's very useful imo, and this is a per-game option.

Not all games will have a profile called Default though, you can rename and delete profiles (though you always have to have at least one afaik, but you can name it anything - my New Vegas default profile is called "Vanilla").

does it make sense to sync/symlink the whole profiles/Default?

I believe so, and it should be already symlinked since profiles is one of the directories constructed from the base directory mentioned earlier. There shouldn't be any need to change the behaviour here, it should work as-is.

From the other comment:

If someone has hints what is required for sharing the list of enabled mods with the game instance of the "global" setup with the "portable" setup any hints welcome (for example merging ModOrganizer.ini parts already is enough?)

I'm not totally sure but I believe the modlist.txt in the profiles folder is how MO2 determines what mods to use and load per-game (since you can have mods installed but de-selected so they are not loaded in that profile), along with other files in this folder like loadorder.txt and such. I don't know too much more than this but maybe that's at least someewhat useful.

I haven't really used portable instances with MO2 but once I have access to a fresh machine with no mod installs setup on, I can give it a whirl.

Okay, that was really long. Tried my best to format it as best as I could. There's a lot here, hopefully useful! Let me know if there's anything I need to explain better or can help with.

sonic2kk commented 2 years ago

Actually, I could probably test portable instances with some other games I have installed that support MO2. If there's anything specific with portable instances you need me to test we can go over that and I can try my best to give some info.

frostworx commented 2 years ago

Thanks for the time you spent into the investigation and of course for the huge summary! You pointed out several important points and one will definitely be a problem again: User configuration within mo2. The main goal is to use the same per-game configuration files from the global instance (always mo2 compatdata) and the corresponding portable instance (always game compatdata). Although I think this is doable with major effort, It will be nearly impossible to protect it from breaking. user of course want to play around with configs and settings, which certainly will break any symlinks or "synced" config files soon or later. (this is a problem windows users apparently don't have - they don't have two windows systems installed simultaneously and sync their configs between different instances) It is preprogrammed that skill3d k1dz will rant more than now.

I understand that the global instance is a feature which should not be removed, but we need a solid and stable solution to keep the configs in sync.

Although I barely use it, I know most configs and their locations of MO2 from writing the implementation, but indeed I did not check if the gui itself offers an option to define the download path. Thanks for the hint 🤓. Means it should be possible to preconfigure the download path for both global and portable installations to the same directory without the need of any symlinks (unfortunately this is the smaller problem, but anyway a step forward)

Thanks again for your help, and: please give some other users the chance to help as well 😀

edit: took a closer look 😏: I think the pointer to the "paths" tab in the mo2 settings and the [Settings] snippet was very useful. It should be the easiest and also the safest way to simply set all of the configurable paths in every portable instance (in the game compatdata) to those used in the global instance _(so basically ~/.config/steamtinkerlaunch/mo2/compatdata/pfx/drivec/users/steamuser/AppData/Local/ModOrganizer/game/*) this should cover pretty much all and only the ModOrganizer.ini itself is left (if needed) edit2: in fact this could be even set automatically for all global instance games, because the paths are default anyway and identical as well.

sonic2kk commented 2 years ago

Thanks again for your help, and: please give some other users the chance to help as well

Ah, you mentioned this in another issue too :smile: Sure, getting feedback from other users is important especially to avoid a scenario where "it works on my machine". The more people that chime in, the better! (Especially since my PC and laptop both run Arch, there's not a lot of "distro variety" in my reports).

It should be the easiest and also the safest way to simply set all of the configurable paths in every portable instance (in the game compatdata) to those used in the global instance

On the subject of this, I went back and did another little test and the position under [Settings] doesn't seem to matter. I scattered them around under that block and MO2 still saw them - I mean, I kind of expected this since all the ini files I've seen before have behaved this way, but wanted to update just to say the position shouldn't matter when writing to the ini file.

And one last thing: Thanks a ton for the major effort on overhauling MO2. It's sounding amazing so far and going to these lengths primarily for other users is super awesome of you :-) Hope we can get more users to give more feedback on how to best implement the overhaul! (One idea maybe - If it wasn't posted already, asking for ideas on the subreddit could be a good way to drum up reports on how to improve MO2 integration with STL/find out some quirks of MO2 and how to workaround/leverage them :smiley:)

frostworx commented 2 years ago

Thanks! I already tried to motivate the sureddit members to help each other, but apparently this didn't help (the few users who try to help, already did so before)

I've got some WIP news for the current (offline) state of the MO2 overhaul 🎉:

This all requires further testing and some improvements, but we're making progress 🤓

Not sure yet how to proceed with user-created ModOrganizer.ini in a game compatdata. I tend to just leave it as-is and just log that it wasn't touched by default. It might be possible to add an option for a migrate, so the user can decide if the paths above should be added to the ini and the found data would be migrated over. might cause more trouble (/support) than benefit though.

frostworx commented 2 years ago

Just pushed the current overhaul state into this mo2-overhaul branch Please test as much as possible :)

sonic2kk commented 2 years ago

Haven't had a chance to test this yet but I think there was a comment earlier about some UI elements being broken with MO2. This could be because of custom Wine themes - DotNet apps can have visual glitches with some Wine themes like the "Light" theme that is enabled by default in GE-Proton (and I think vanilla Wine+TKG wine as well). You can change this in Winecfg for your Proton version and it's under the "Desktop Integration" tab.

Regular Proton direct from Valve, as far as I know, comes with this theme disabled and actually I don't even know if they ship that theme at all. For regular Proton the Wine theme should be "(none)".

This can also be fixed, in my experience, by setting a custom theme in the MO2 settings (separate from the Wine theme - This is for setting the theme that MO2 itself sets).

If anyone experiences some issues with MO2 elements being black boxes or flickering, changing the MO2 theme in the MO2 settings or Wine theme with Winecfg may help resolve it.

Sorry, I couldn't find the exact issue this was mentioned in, but I remembered there was some talk of this and at work it hit me that this might be able to fix that issue.

jake770321 commented 2 years ago

Tried it out real quick and everything other than nxm links seems to be working for me. I was able to create a portable MO2 instance, install a mod, and run the game without issue. The nxm link not working may very well be because of my trainwreck of a linux install as even the "visit the nexus" button wasn't working for me until i installed a different browser via flatpak which suddenly made that button work? :confused: I've also never used the sudo make install command before and had to copy over the lang folder to even get it to run so you can safely ignore me running into problems with nxm unless someone else does the same.

I'll try again properly in a VM as well as also try to copy over a wabbajack install (which is basically a portable mo2 instance) as soon as i can. I've never used nxm links anyway as i've always just copied the mod directly from the downloads folder into the mo2 downloads tab and can confirm that this works perfectly with steamtinkerlaunch.

frostworx commented 2 years ago

Thanks for testing, @jake770321! Would be great if you could provide a log after you clicked a nxm link - it might have found what could have gone wrong.

jake770321 commented 2 years ago

Not entirely sure which log to include so hopefully this is the correct one. I've replaced any potentially personally identifiable info with *****. This is on a clean steamtinkerlaunch install using the latest version (at the time of posting) and the only setting changed being the mod organizer 2 mode set to gui. I've used the portable mode for MO2 (as well as a fresh firefox profile) and tried opening a nxm link before and after hitting the 'Associate with "Download with manager" links' button in MO2 itself but got the exact same log both times.

This is from /.config/steamtinkerlaunch/logs/steamtinkerlaunch/id/31337.log

Thu 13 Oct 03:14:14 BST 2022 INFO - initAID - Set AID to PLACEHOLDERAID '31337' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Language from command line is 'english' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - SYSTEMSTLCFGDIR is '/usr/share/steamtinkerlaunch' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Command line language 'english' is no file - trying to find its absolute path Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Loading found user-installed /home/*****/.config/steamtinkerlaunch/lang/english.txt Thu 13 Oct 03:14:14 BST 2022 INFO - getCurrentCommandline - No Proton in command line arguments 'mo2 u nxm://stardewvalley/mods/1915/files/61370?key=*****' Thu 13 Oct 03:14:14 BST 2022 INFO - saveOrgVars - Storing some original variables to restore them later Thu 13 Oct 03:14:14 BST 2022 INFO - emptyVars - Emptied some original variables as they slowdown several system calls when started from steam Thu 13 Oct 03:14:14 BST 2022 INFO - emptyVars - Set $PATH to '/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/snap/bin:' Thu 13 Oct 03:14:14 BST 2022 START - ######### Initializing Game Launch 31337 using SteamTinkerLaunch v11.12.20221012 ######### Thu 13 Oct 03:14:14 BST 2022 INFO - main - Starting with a clean log Thu 13 Oct 03:14:14 BST 2022 INFO - main - Start creating default configs Thu 13 Oct 03:14:14 BST 2022 INFO - createDefaultCfgs - START Thu 13 Oct 03:14:14 BST 2022 INFO - loadLanguage - First load the default language 'english' to make sure all variables are filled Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Language from command line is 'english' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - SYSTEMSTLCFGDIR is '/usr/share/steamtinkerlaunch' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Command line language 'english' is no file - trying to find its absolute path Thu 13 Oct 03:14:14 BST 2022 INFO - loadLangFile - Loading found user-installed /home/*****/.config/steamtinkerlaunch/lang/english.txt Thu 13 Oct 03:14:14 BST 2022 INFO - loadLanguage - Loading STLLANG from '/home/*****/.config/steamtinkerlaunch/global.conf' Thu 13 Oct 03:14:14 BST 2022 INFO - loadLanguage - STLLANG from '/home/*****/.config/steamtinkerlaunch/global.conf' is 'english' Thu 13 Oct 03:14:14 BST 2022 INFO - createDefaultCfgs - setSteamPaths: Thu 13 Oct 03:14:14 BST 2022 INFO - setSteamPaths - Reading Steam Path variables from '/dev/shm/steamtinkerlaunch/steampaths.txt' Thu 13 Oct 03:14:14 BST 2022 INFO - getGameOS - Starting game OS detection Thu 13 Oct 03:14:14 BST 2022 INFO - getGameOS - STEAM_COMPAT_DATA_PATH is not defined, so this is either a Linux Game or no game was started at all Thu 13 Oct 03:14:14 BST 2022 INFO - listAllSettingsEntries - '/dev/shm/steamtinkerlaunch/setentries.txt' already exists - nothing to do Thu 13 Oct 03:14:14 BST 2022 INFO - createDefaultCfgs - STOP Thu 13 Oct 03:14:14 BST 2022 SKIP - listSteamLibraries - not recreating already available '/dev/shm/steamtinkerlaunch/SteamLibraries.txt' Thu 13 Oct 03:14:14 BST 2022 INFO - setSteamLibraryPaths - STEAM_COMPAT_LIBRARY_PATHS set to '/home/*****/.local/share/Steam/steamapps:/mnt/*****/Steam/steamapps' Thu 13 Oct 03:14:14 BST 2022 INFO - setSteamLibraryPaths - STEAM_COMPAT_MOUNTS set to '/home/*****/.local/share/Steam/steamapps/common/SteamLinuxRuntime_sniper:/home/*****/.local/share/Steam/steamapps/common/SteamLinuxRuntime_soldier:/home/*****/.local/share/Steam/steamapps/common/Steamworks Shared:/mnt/*****/Steam/steamapps/common/SteamLinuxRuntime_soldier' Thu 13 Oct 03:14:14 BST 2022 INFO - setSteamLibraryPaths - STEAM_COMPAT_TOOL_PATHS set to '/home/*****/.local/share/Steam/steamapps/common/SteamLinuxRuntime_sniper:/home/*****/.local/share/Steam/steamapps/common/SteamLinuxRuntime_soldier:/home/*****/.local/share/Steam/steamapps/common/Steamworks Shared:/mnt/*****/Steam/steamapps/common/SteamLinuxRuntime_soldier' Thu 13 Oct 03:14:14 BST 2022 INFO - main - Checking internal dependencies: Thu 13 Oct 03:14:14 BST 2022 INFO - setAwkBin - Found 'gawk' as an 'awk' variant. It should work without any issues, because 'gawk' was tested completely Thu 13 Oct 03:14:14 BST 2022 INFO - checkIntDeps - Result of version check for yad binary '/usr/bin/yad' is '12.0' Thu 13 Oct 03:14:14 BST 2022 INFO - main - Initializing first Proton: Thu 13 Oct 03:14:14 BST 2022 INFO - initOldProtonArr - Creating an initial array with available Proton versions using the file '/dev/shm/steamtinkerlaunch/ProtonCSV.txt' which was created during a previous run Thu 13 Oct 03:14:14 BST 2022 INFO - initFirstProton - Initializing Proton Thu 13 Oct 03:14:14 BST 2022 INFO - initFirstProton - Initial Proton version 'FIRSTUSEPROTON' from '/home/*****/.config/steamtinkerlaunch/default_template.conf' is 'proton-7.0-4' Thu 13 Oct 03:14:14 BST 2022 INFO - main - Initializing default window resolution Thu 13 Oct 03:14:14 BST 2022 INFO - setInitWinXY - Using '1920x1080' from config '/dev/shm/steamtinkerlaunch/defres.txt' Thu 13 Oct 03:14:14 BST 2022 INFO - main - Checking command line: incoming arguments 'mo2 u nxm://stardewvalley/mods/1915/files/61370?key=*****' Thu 13 Oct 03:14:14 BST 2022 INFO - setMO2Vars - No game wineprefix found in env, so using a global instance of 'mo2' Thu 13 Oct 03:14:14 BST 2022 INFO - setMO2Vars - Preparing Proton variables for a global mo2 instance Thu 13 Oct 03:14:14 BST 2022 INFO - setMO2Vars - Using GE-Proton7-14 for ModOrganizer Thu 13 Oct 03:14:14 BST 2022 INFO - dlMod2nexurl - Attempted to download Url 'nxm://stardewvalley/mods/1915/files/61370?key=*****' for game 'stardewvalley', but the source script '/home/*****/.config/steamtinkerlaunch/mo2/dldata/stardewvalley.conf' for portable mo2 is missing - trying to start a global ModOrganizer Thu 13 Oct 03:14:14 BST 2022 ERROR - dlMod2globnexurl - Could not find a valid ModOrganizer instance for 'stardewvalley' - giving up

EDIT: Not sure if this is relevant but i'll include just in case: /.config/steamtinkerlaunch/logs/gamelaunch/id/413150.log The formatting for this one broke on github so i've posted it here: https://privatebin.net/?259c442ef5780d72#D4NgTU1yyvgQhWKqrpzhEEE9ksvqZsYjVXaWJzz9f9JT The last three lines are what i get when clicking on the "visit nexus" button in MO2 itself.

frostworx commented 2 years ago

Thanks for the log! Please attach it the next time, as the formatting is badly formatted. It helped anyway, as the error is at the end of the log:

Attempted to download Url 'nxm://stardewvalley/mods/1915/files/61370?key=*****' for game 'stardewvalley', but the source script '/home/*****/.config/steamtinkerlaunch/mo2/dldata/stardewvalley.conf' for portable mo2 is missing - trying to start a global ModOrganizer Thu 13 Oct 03:14:14 BST 2022 ERROR - dlMod2globnexurl - Could not find a valid ModOrganizer instance for 'stardewvalley' - giving up

The portable file '/home/*****/.config/steamtinkerlaunch/mo2/dldata/stardewvalley.conf' is created when you start stardewvalley from steam with mo2 enabled. The global.conf is created when you launch steamtinkerlaunch mo2 start Please be so kind and try this and report back.

Thanks for the XPCOM error as well. Not sure if it is related to wine/proton not opening the native browser, but I'll keep this in mind when troubleshooting.

sonic2kk commented 2 years ago

Just FYI because I ran into this while testing: It seems like MO2 only supports modding Windows versions of games. From my skim of the log it looks like this might be the Linux build of Stardew Valley. For testing make sure you're using the Windows build as I wasn't able to get MO2 to correctly mod the Linux version of games 🙂

I'm almost certain this is on the MO2 side - Not STL or even the mods (though some mods do only support certain platforms, it seems rare for Stardew Valley from my VERY brief understanding).

The reason I think it's MO2 is because if you try to manually set the instance path to the Stardew Valley Linux folder, it can't see the game. It's probably looking for an executable. It would be cool if MO2 actually supported this (maybe if you symlink the Linux executable to have the Windows exe name MO2 might see it, but this could have various other complications!)

frostworx commented 2 years ago

Starting game OS detection Thu 13 Oct 03:14:14 BST 2022 INFO - getGameOS - STEAM_COMPAT_DATA_PATH is not defined, so this is either a Linux Game or no game was started at al good catch!

jake770321 commented 2 years ago

I'm pretty sure I was running the windows version of the game as i was still able to launch and play the game through MO2. I've gone ahead and tried Fallout: New Vegas (portable MO2) as well since it doesn't have a native linux version so there's less room for mistake on my end and i've gotten the exact same error. Log: 31337.log

I ran the steamtinkerlaunch mo2 start command and created a New Vegas MO2 global instance(it didn't detect new vegas automatically when doing it this way so i had to hit browse and add it manually but that's not a big deal). I'm not able to launch the game when using this command as i get a "This program requires steam" prompt and neither options get the game to launch. Weirdly enough, the "Visit Nexus" button worked when launched like this but nothing changed when it came clicking an nxm link. Here is the log when using the above command to create a global MO2 instance and then clicking on a nxm link: 31337.log I've also included the log from /.config/steamtinkerlaunch/logs/steamtinkerlauch/title/ Fallout New Vegas.log

Personally, I don't think nxm link support is that important as just dragging files from your download folder into the MO2 download tab is already incredibly easy and straightforward. The big thing (for me) with this mo2 overhaul is support for portable mode as this opens up support for wabbajack installs and i'll get right on testing that next (unless there's something else you'd like me to test first)

Edit: I should also add that sometimes the MO2 ui bugs out when returning to it after closing the game but this is easily fixed by restarting MO2 and I don't think it's a steamtinkerlaunch issue.

frostworx commented 2 years ago

Thanks for the further tests! Hmm, New Vegas should be auto-detected, but it is not detected as compatible: SKIP - manageMO2GInstance - '22380' is no supported ModOrganizer game This means that the search for it in the bundled helper file "$MO2GAMES" is not successful. Please share the output of grep -m1 22380 /usr/share/steamtinkerlaunch/misc/mo2games.txt

(just added some more logging)

jake770321 commented 2 years ago

"New Vegas";"22380";"New Vegas";"newvegas"

It was automatically detected when launching New Vegas with steamtinkerlaunch set as the compatibility tool: just not when running steamtinkerlaunch mo2 start from the terminal.

frostworx commented 2 years ago

Hm, can't reproduce this. just ran steamtinkerlaunch mo2 start with a clean ~/.config/steamtinkerlaunch and /dev/shm/steamtinkerlaunch and everything worked as expected

sonic2kk commented 2 years ago

The branch for this is still open and pending contributions from any testers to see how stable it is. Once we have more feedback I can take a look at merging this in.

sonic2kk commented 2 years ago

Tentatively planning to test the mo2-overhaul branch with my own modded setups sometime on the 25th. Any tests from other users before then would be much appreciated! The branch is forked off from an older version of STL, so if you are testing you probably won't want to use that branch as a full-time replacement for your regular STL usage :-)

If all goes to plan, user feedback is good (or silent :stuck_out_tongue:) and my existing mod setups still work as expected (from glancing at the branch, they should), I'm hoping to get this merged before the end of the month, in time for broader testing before the next release.

If there are issues and we have to hold off on merging, a couple of fixes from this branch (namely the Wine downloading fixes) will be pulled in for the next release.

har-nick commented 2 years ago

Hi there! Two issues I've noticed getting MO2 to work.

1) STL doesn't seem to recognise any lang file besides english.txt. I'm unsure if I'm doing something wrong as copying german.txt and renaming it to english.txt still shows English strings. Setting the lang= option works, but has the same effect.

2) Anything to do with MO2 throws an error:

cat: /usr/share/steamtinkerlaunch/misc/mo2games.txt: No such file or directory

Again, I'm unsure if I'm doing something wrong here. I'm just running STL directly from the pulled folder after checking out the overhaul branch.

frostworx commented 2 years ago

https://github.com/sonic2kk/steamtinkerlaunch#how-do-i-install-it

sonic2kk commented 2 years ago

Hi! Thanks for testing the MO2 overhaul branch!

Your issue with the language files could be because the MO2 branch is behind master, and likewise your MO2 Games problem is probably also because it's behind master. I guess this doesn't happen for you on master from your other comments :-)

So to test the MO2 branch, you have to use sudo make install to install MO2, taking care to remove any redundant package you may have installed from your package manager. If you're on Steam Deck it should just work by double clicking the script.

I should add a more explicit note somewhere, but to test development branches it's best to do a root install with sudo make install. When you're done you can remove the root install with sudo make uninstall (you may need to go back to the master branch to test this).

Good luck!