Perchik71 / Creation-Kit-Platform-Extended

A collection of modifications, enhancements, and reverse engineered resources for Creation Kit by Bethesda.
GNU General Public License v3.0
70 stars 9 forks source link

MO2 incompatibility from 0.2 build 672 onwards findings #54

Open naaitsab opened 4 months ago

naaitsab commented 4 months ago

As mentioned before using the newer versions of CKPE with MO2 causes a lot of crashes on loading mainly larger ESP/ESM files. For example this thread: https://github.com/Perchik71/Creation-Kit-Platform-Extended/issues/53

As per request on the Nexus thread, I used the 'filter' system introduced in 0.3 build 368 to filter out fixes/elements until it was stable. As this has the least changes between 0.2 672 I'm also testing it on that specific version.

I found that the following combination greatly increases the chance of not crashing into a C0000005 error: Load Optimization Main Window Memory Leak class BSString Memory Leak In Peview Window Memory Manager (place these lines in a txt file CreationKitPlatformExtendedFilter.txt in the game directory before starting the CK)

As perchik isn't using MO2. Can other people try these filters to see if it helps with stability on their PC? For me it's still not 100% stable but a lot better, albeit a bit slower loading. If the cause is linked to these modules maybe we can troubleshoot them and see if we can make it compatible with MO2 as that is used a lot by modders.

robertgk2017 commented 4 months ago

sure your using a portable instance, running it as admin, have a Defender exclusion setup, and dont have anything even remotely related to skyrim and modding in Program Files or any other system folder?

Im working on a 315MB + ESM and ive yet to have any issues.

Perchik71 commented 4 months ago

To be objective, Memory Leak class BSString was added version newer that 0.2 672. This bstring.append. Basically, he can only disable this patch. The truth is not to be surprised about the increase in RAM consumption, there leak.

naaitsab commented 4 months ago

sure your using a portable instance, running it as admin, have a Defender exclusion setup, and dont have anything even remotely related to skyrim and modding in Program Files or any other system folder?

Im working on a 315MB + ESM and ive yet to have any issues.

Nothing portable Running exe's from MO2, or MO2 itself as admin is not advisable. AV exclusion/disabling does not help with crashing on the 000005 error Data folder is clean of files except the Bethesda provided files. That's what MO2 is for.

naaitsab commented 4 months ago

To be objective, Memory Leak class BSString was added version newer that 0.2 672. This bstring.append. Basically, he can only disable this patch. The truth is not to be surprised about the increase in RAM consumption, there leak.

It could cause issues, I will monitor the memory usage. Do the patches force any kind of memory allocation or expansion beyond the original exe? The WinDBG reports always refer to memory pointers/calls that are invalid. As these filters that appear to help with stability seem seems like memory related?

robertgk2017 commented 4 months ago

sure your using a portable instance, running it as admin, have a Defender exclusion setup, and dont have anything even remotely related to skyrim and modding in Program Files or any other system folder? Im working on a 315MB + ESM and ive yet to have any issues.

Nothing portable Running exe's from MO2, or MO2 itself as admin is not advisable. AV exclusion/disabling does not help with crashing on the 000005 error Data folder is clean of files except the Bethesda provided files. That's what MO2 is for.

Not using a portable instance means a lot of mo2s files will be stored inside windows system folders, this is ill adviseable as Defender/UAC are very over protective over these folders. Hence, the recommendation to NEVER store ANYTHING even remotely related to the game and modding tools anywhere near those system folders. That includes Steam, the game itself, mo2, you're other tools, etc. an mo2 portable instance will store everything directly inside the mo2 install folder, which itself should be under a new top folder from the root of your Drive.

Running mo2 as admin, which by extension runs the other programs launched from it as admin, is always adviseable, that will prevent windows from blocking/screwing up any of the modding tools trying to work.

AV exclusions do in fact help, they should be added for the game exe, skse exe, mo2 exe, any other tool exes, and the USVFS dlls that ship with mo2, as well as a folder exclusion for any top folders you have setup for holding your modding stuff.

The Data folder being clean of files has absolutely nothing to do with my comment, that is completely irrelavent. If you want your modding setup to work, guaranteed, follow the advice im giving to you, that i give to absolutely every other user in all conceivable circumstances.

naaitsab commented 4 months ago

sure your using a portable instance, running it as admin, have a Defender exclusion setup, and dont have anything even remotely related to skyrim and modding in Program Files or any other system folder? Im working on a 315MB + ESM and ive yet to have any issues.

Nothing portable Running exe's from MO2, or MO2 itself as admin is not advisable. AV exclusion/disabling does not help with crashing on the 000005 error Data folder is clean of files except the Bethesda provided files. That's what MO2 is for.

Not using a portable instance means a lot of mo2s files will be stored inside windows system folders, this is ill adviseable as Defender/UAC are very over protective over these folders. Hence, the recommendation to NEVER store ANYTHING even remotely related to the game and modding tools anywhere near those system folders. That includes Steam, the game itself, mo2, you're other tools, etc. an mo2 portable instance will store everything directly inside the mo2 install folder, which itself should be under a new top folder from the root of your Drive.

Running mo2 as admin, which by extension runs the other programs launched from it as admin, is always adviseable, that will prevent windows from blocking/screwing up any of the modding tools trying to work.

AV exclusions do in fact help, they should be added for the game exe, skse exe, mo2 exe, any other tool exes, and the USVFS dlls that ship with mo2, as well as a folder exclusion for any top folders you have setup for holding your modding stuff.

The Data folder being clean of files has absolutely nothing to do with my comment, that is completely irrelavent. If you want your modding setup to work, guaranteed, follow the advice im giving to you, that i give to absolutely every other user in all conceivable circumstances.

MO2 uses instances that are outside of the main application installation directory. This is the default behaviour. If you call this portable then yes, it's running portable. Installing the core files of MO2, so not the instance inside program files makes little difference if you make some minor changes. The only difference is that the NTFS permissions on the program files folder are stricter than some folder outside. By default it gives the user running Windows only read access to the folder after installation. Unless elevated by UAC. Writing is done to the users appdata or registry to protect the integrity of the application files, this is Windows applications 101. There is nothing fancy or protected about the program files directory outside of this. Not sure why people keep treating it as some kind of scary folder without fact checking. The only reason this could be an issue if you don't start MO2 as admin at least once if it's installed in Program Files. So it can't change/write any files inside it's own dir. Which is not required if you move it outside of program files. For ease of backup and not having to modify the NTFS permissions I have my MO2 installed in another folder and have the instances separated. Which is still best practice as it will allow the users to start the application without further steps that can go wrong.

Running MO2 as admin is not advisable as it calls other executables which cause issues with the passing of the elevation. This is only required if the NTFS permissions are not modified to allow the MO2 exe write access to it's own folder. Nothing more, nothing less. MO1 did not handle passing elevations at all, hence the requirement to install it outside of program files. Running apps as admin unnecessary is never advisable on Windows from a security standpoint. Especially if you are running modded DLL's that can easily infect your computer or break stuff.

USVFS gets triggered by some AV's as it's in essence a modified DLL to allow symlinks to work outside of Microsoft's own system-wide solution, as it allows an instance per app. So it can be used to hide malicious activity. Hence why you need to exclude the DLL in the install directory of MO2 on some AV's. If any patch or addition to the CK itself is causing AV's to trigger there is probably a good reason why. It's really inadvisable to blanket whitelist entire folders. If you use a decent AV, and if the crashes where due to AV then a log is always generated that it did something. AV does nothing "magickly" without logging the action. i.e. kill the CK.exe. AV can slow down processing as it monitors all activity. So for example updating a Nemesis build can get bogged down due to the high I/O. The times that AV's redirected memory 'traffic' and thus causing instability is long gone. It's a monitoring agent that only acts when it gets triggered by a definition rule.

I've been modding Skyrim since 2012 and have more than 3300 hours of CK LE, FO4 and SE combined clocked. Starting with MO1 somewhere in 2013. So I'm inclined to say that I know my way around modding Skyrim with these tools a fair bit. While there is nothing wrong with giving tips. Outside of the placement of the MO2 folders and USVFS DLL AV exclusion, most information is dated, inadvisable or have a placebo effect while compromising security. The issue at play here is that the later builds of CKPE causes certain (memory) issues combined with MO2 while loading large mods/sources. Which causes the CK itself to crash. Probably due to the USVFS used by MO2 as it does not seem to happen on Vortex or 'manual' installs using mod files in the Data directory as a test. On those CKPE runs without issue. Also keep in mind, this is not only on my PC/workflow but multiple cases on Discord servers, Github and Nexus. So the chance of everyone 'doing it wrong' is extremely unlikely.

robertgk2017 commented 4 months ago

No no, in mo2 theres 2 explicit types of instances, global or portable. Open the instance manager it lists what kind you're using. Global puts stuff in a system folder, portable puts them in the mo2 install folder. When you make a new one the prompt asks which kind you want to use.

Portable is vastly preferred due to the files being inside the mo2 install folder and not thrown in a windows system folder.