synthesis fail to overwrite previous patch

[Niscent0]

Hello, here's the issue i encountered on windows 10 with the latest synthesis version. upon finishing the running of each patchers, synthesis will be locked out of accessing the .esp it's outputting to because it's already being accessed by another program, which is... synthesis itself (according to my system at least). i figured a way to get the patcher to run properly by simply deleting the output esp manually while synthesis is fully closed. this is still fairly inconvenient and i believe would warrant a fix if it's not some issue on my part.

here's the error log from synthesis:

System.IO.IOException: The process cannot access the file 'E:\GAME\The Elder Scrolls - Skyrim - Special Edition\Data\Synthesis.esp' because it is being used by another process.
   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
   at System.IO.FileInfo.CopyTo(String destFileName, Boolean overwrite)
   at System.IO.Abstractions.FileInfoWrapper.CopyTo(String destFileName, Boolean overwrite)
   at Noggog.IFileSystemExt.DeepCopy(IDirectory system, DirectoryPath from, DirectoryPath to, Boolean overwrite) in D:\a\CSharpExt\CSharpExt\Noggog.CSharpExt\Extensions\IFileSystemExt.cs:line 165
   at Synthesis.Bethesda.Execution.Running.Runner.MoveFinalResults.Move(FilePath finalPatch, DirectoryPath outputPath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\MoveFinalResults.cs:line 50
   at Synthesis.Bethesda.Execution.Running.Runner.RunAGroup.Run(IGroupRun groupRun, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\RunAGroup.cs:line 68
   at Synthesis.Bethesda.Execution.Running.Runner.RunAllGroups.Run(IGroupRun[] groups, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\RunAllGroups.cs:line 36
   at Synthesis.Bethesda.Execution.Running.Runner.ExecuteRun.Run(IGroupRun[] groups, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\ExecuteRun.cs:line 54
   at Synthesis.Bethesda.GUI.Services.Profile.Running.ExecuteGuiRun.Run(IEnumerable`1 groupRuns, PersistenceMode persistenceMode, Boolean localize, Boolean utf8InEmbeddedStrings, Language targetLanguage, CancellationToken cancel) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.GUI\Services\Profile\Running\ExecuteGuiRun.cs:line 47
   at Synthesis.Bethesda.GUI.ViewModels.Profiles.Running.RunVm.<Run>b__39_1(Unit _) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.GUI\ViewModels\Profiles\Running\RunVm.cs:line 209```

[Noggog]

Hmm, this was an issue early on in Synthesis' development, but I think it's been mostly fixed up to not do that anymore.

I do know generally Synthesis is able to rerun itself multiple times without getting itself locked out, ala this video I just took:

https://user-images.githubusercontent.com/24981326/236700446-be7fe0b3-9b2e-496a-8498-29a10719a559.mp4

I don't think generally people have had to do the deletion step like you're mentioning, so I think something abnormal is at play for you.

How did you check/confirm that it's Synthesis itself locking its own files? Do you have any Antivirus besides windows defender?

[Niscent0]

i totally forgot to mention, until recently i had no issue rerunning synthesis again and again (or at least didn't notice). i only use windows defender so no issue on that side. when i get back to synthesis main menu after failure and try to delete synthesis.esp, i get the "the action can't be completed because the file is open in Synthesis" popup from windows. after closing synthesis completely i can delete it successfully.

new info though: i didn't notice, at first, but after trying some more usual esoteric diagnostic attempts, i found out the issue does not present itself when i'm first starting synthesis. so i can rerun the patchers without deleting the esp if i close and restart synthesis, but it gives me the error message when i run it twice in a row like you did. which makes much more sense.

i presume for some reason my synthesis doesn't close its file access after patching, and the next attempt at opening the file returns an error. and obviously upon closing synthesis, the system cleanup all allocations and file opened so it works when i restart it. it could also be an actual overlook in the code were synthesis is just not closing the file, and it works in other people's environments because their system closes the file automatically (automatic leaks and loose references cleaning maybe). so i would be the only one getting the error because my system does not do that cleanup. i have no clue, just throwing ideas, i didn't even look into the sources yet.

[Noggog]

Are there any other errors in the logs?

Also, what sort of grouping setup are you running? Just one large synthesis group? Or a few synthesis groups? If there's a few, are you running each group individually, or running the whole pipeline via the run all button at the bottom left?

Shoot up the logs on a run where you did the patching twice and it locked out, maybe something to spot inside

[Niscent0]

no other error or warning in the logs of the individual patchers, the error log is still similar. if you want i'll add the logs of each patchers too, i didn't do it because i didn't want to clog my message. and just to be sure i'm not causing any misunderstanding, what i've been calling "error log", is what's displayed after clicking on the red '!' that appears at the top of the left side colon after the writing fails.

System.IO.IOException: The process cannot access the file 'E:\GAME\The Elder Scrolls - Skyrim - Special Edition\Data\Synthesis.esp' because it is being used by another process.
   at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
   at System.IO.FileInfo.CopyTo(String destFileName, Boolean overwrite)
   at System.IO.Abstractions.FileInfoWrapper.CopyTo(String destFileName, Boolean overwrite)
   at Noggog.IFileSystemExt.DeepCopy(IDirectory system, DirectoryPath from, DirectoryPath to, Boolean overwrite) in D:\a\CSharpExt\CSharpExt\Noggog.CSharpExt\Extensions\IFileSystemExt.cs:line 165
   at Synthesis.Bethesda.Execution.Running.Runner.MoveFinalResults.Move(FilePath finalPatch, DirectoryPath outputPath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\MoveFinalResults.cs:line 50
   at Synthesis.Bethesda.Execution.Running.Runner.RunAGroup.Run(IGroupRun groupRun, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\RunAGroup.cs:line 68
   at Synthesis.Bethesda.Execution.Running.Runner.RunAllGroups.Run(IGroupRun[] groups, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\RunAllGroups.cs:line 36
   at Synthesis.Bethesda.Execution.Running.Runner.ExecuteRun.Run(IGroupRun[] groups, CancellationToken cancellation, DirectoryPath outputDir, RunParameters runParameters, Nullable`1 sourcePath) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.Execution\Running\Runner\ExecuteRun.cs:line 54
   at Synthesis.Bethesda.GUI.Services.Profile.Running.ExecuteGuiRun.Run(IEnumerable`1 groupRuns, PersistenceMode persistenceMode, Boolean localize, Boolean utf8InEmbeddedStrings, Language targetLanguage, CancellationToken cancel) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.GUI\Services\Profile\Running\ExecuteGuiRun.cs:line 47
   at Synthesis.Bethesda.GUI.ViewModels.Profiles.Running.RunVm.<Run>b__39_1(Unit _) in D:\a\Synthesis\Synthesis\Synthesis.Bethesda.GUI\ViewModels\Profiles\Running\RunVm.cs:line 209```

i'm running a single group with only 3 patchers: SynHelmsWithCirclets, Requiem-Experience, and Actor-Edge-Glow-Remover. just to be sure, i tried running it twice in a row with the group button, fully closed and restarted synthesis, then twice again with the run all button, and got the same result.

just to mention it too, the requiem-experience one has his synthesis and mutagen versions set to "match" rather than "latest" because it's not updated and break with later versions.

[Noggog]

There's a logs folder next to the exe, with an Everything.txt that should have the goods

[Niscent0]

here it is: Everything.txt

[Noggog]

Hmm, odd situation, for sure.

Just for some more detail, the code to write out to the data folder to make the file that's locked isn't really a stream open that might be mistakenly not disposed. Rather, it writes to a temp area, and then calls CopyTo (via the System.IO.Abstractions library). So best I can guess if it's on Synthesis' side is that somehow the engine is perhaps opening a stream after that somehow?

Have you tried running Synthesis outside of Mo2/Vortex on a simple derpy load order, and seeing if that experiences the same locking?

[Niscent0]

i'm already running synthesis outside of vortex (and with vortex closed). not sure how to manufacture a derpy load order to test that though. i can copy the base game esm on the side, but i don't know how to make a different load order file and point synthesis to it.

[Noggog]

Oh, not too familiar with exactly how you're using it, but Vortex/Mo2 can do Virtual File Systems where they overlay the mods on top of the Data folder. So running Synthesis outside Mo2 usually results in a "derpy" load order as it's just the base esms without all the mods overlaid on top. But you're saying all your mods actually live in your Data folder, so running outside of Vortex results in Synthesis properly patching against your full load order?

[Niscent0]

to my knowledge, i'm just using the default hardlink deployment method. so once vortex has created the links in file system, "they just work (tm)". i always just ran patchers and external tools separately with vortex off.