[BUG] Tries to open ROM file 3 times

[masagrator]

It seems that code has bug that tries to open original rom file 3 times.

https://github.com/DCNick3/ShinDataUtil/blob/master/Compression/ShinRomOperations.cs#L25-L26 https://github.com/DCNick3/ShinDataUtil/blob/master/Decompression/FileReadableGameArchive.cs#L31-L32

I never programmed in dotnet, so dunno how to fix this. That's why no pull request from me. I have managed to hack it by doing 2 copies of rom file and providing each one separately.

[DCNick3]

Yeah, it does open in three times. Why do you consider it a bug?

[masagrator]

When trying to run this untouched on my PC, I'm getting an error that "this file cannot be opened because it's used by another process". Compiled with Visual Studio 2019, running on W10.

Only not allowing to open files more than one time solves issue.

[DCNick3]

Oh, that's weird, seems like windows tries to open it without sharing enabled. Works fine on GNU/Linux.

Looks like it's a "feature" of desktop .NET Framework: MemoryMappedFile.CreateFromFile overload with file name opens it with FileShare.None mode, while dotnet core one does set FileShare.Read. I think that using an overload which accepts FileStream and passing the already open one would do the trick.

[DCNick3]

c1348069f501e5f37f398c45a8c2ab371f444522 should fix it

Does it?

[masagrator]

Now I get

C:\Users\Admin\Downloads\ShinDataUtil\bin\Debug\netcoreapp5.0>shindatautil rom-replace-file patch.rom output.snr main.snr patch2.rom
Unhandled exception. System.UnauthorizedAccessException: Access to the path is denied.
   at System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle memMappedFileHandle, MemoryMappedFileAccess access, Int64 offset, Int64 size)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewAccessor(Int64 offset, Int64 size, MemoryMappedFileAccess access)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewAccessor(Int64 offset, Int64 size)
   at ShinDataUtil.Decompression.FileReadableGameArchive..ctor(FileStream fileStream) in C:\Users\Admin\Downloads\ShinDataUtil\Decompression\FileReadableGameArchive.cs:line 42
   at ShinDataUtil.Decompression.FileReadableGameArchive..ctor(String path) in C:\Users\Admin\Downloads\ShinDataUtil\Decompression\FileReadableGameArchive.cs:line 31
   at ShinDataUtil.Compression.ShinRomOperations.ReplaceFile(String inromPath, Stream outrom, Stream injfile, String targetName) in C:\Users\Admin\Downloads\ShinDataUtil\Compression\ShinRomOperations.cs:line 26
   at ShinDataUtil.Program.RomReplaceFile(ReadOnlySpan`1 args) in C:\Users\Admin\Downloads\ShinDataUtil\Program.cs:line 420
   at ShinDataUtil.Program.ActionList.Execute(String actionname, ReadOnlySpan`1 args) in C:\Users\Admin\Downloads\ShinDataUtil\Program.cs:line 511
   at ShinDataUtil.Program.MainWrap(String[] args) in C:\Users\Admin\Downloads\ShinDataUtil\Program.cs:line 542
   at ShinDataUtil.Program.Main(String[] args) in C:\Users\Admin\Downloads\ShinDataUtil\Program.cs:line 549

in both admin and non admin mode

[DCNick3]

Weeeird. Looks like it's time to spin up the Windows virtual machine..

[DCNick3]

Looks like I will even have to reinstall it, as visual studio updater is causing BSODs for some reason =(

[DCNick3]

@masagrator does still cause an issue to you?

[masagrator]

Yup.

Unhandled exception. System.UnauthorizedAccessException: Access to the path is denied.
   at System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle memMappedFileHandle, MemoryMappedFileAccess access, Int64 offset, Int64 size)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewAccessor(Int64 offset, Int64 size, MemoryMappedFileAccess access)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewAccessor(Int64 offset, Int64 size)
   at ShinDataUtil.Decompression.FileReadableGameArchive..ctor(FileStream fileStream) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Decompression\FileReadableGameArchive.cs:line 42
   at ShinDataUtil.Decompression.FileReadableGameArchive..ctor(String path) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Decompression\FileReadableGameArchive.cs:line 31
   at ShinDataUtil.Program.ExtractAllFiles(ReadOnlySpan`1 args, Boolean decode, Boolean dryRun) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Program.cs:line 78
   at ShinDataUtil.Program.<>c.<MainWrap>b__16_0(ReadOnlySpan`1 args_) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Program.cs:line 520
   at ShinDataUtil.Program.ActionList.Execute(String actionname, ReadOnlySpan`1 args) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Program.cs:line 511
   at ShinDataUtil.Program.MainWrap(String[] args) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Program.cs:line 542
   at ShinDataUtil.Program.Main(String[] args) in C:\Users\Admin\Documents\Switch\Higurashi\ShinDataUtil2\ShinDataUtil\Program.cs:line 549

[DCNick3]

Will try to install the visual studio to debug it once again, but it's behaving kinda funky, dunno if I will succeed =)

[DCNick3]

Finally succeeded, can reproduce.

Will proceed to debugging...

[DCNick3]

Should be fixed by b7098e376425ba40c493dc9be8b1089c63058e3e

It appears that even if you have opened a memory mapped file in read-only by default you will try to create a Read-Write accessor. Whatever...

[DCNick3]

Btw, what is your use-case for the tool?

[masagrator]

Well, I'm using other script to pack files to patch.rom as your tool is not usable in cmd when trying to pack more files because of cmd characters limitation (and constantly updating command would be pretty annoying in long term).

So for now I'm using it solely for (de)compiling scenario.

I will try to check tomorrow if commit fixed issue on my side.

[masagrator]

And if you are asking about if it's used in any kind of project, then well... https://gbatemp.net/threads/translation-for-higurashi-switch-game.513201/page-2#post-9487846

[DCNick3]

Hm, good to know.

The program is indeed quite rough on the user interface side, I was more focused on the internals =)

As for the compilation/decompilation of the scenario: it mostly works, but there are definitely some problems regarding its correctness. I was trying to make sure that the decompilation + compilation gives the exact same result, but it currently does not. I fixed some bugs on that side, but there are most probably more lying around.

[DCNick3]

https://gbatemp.net/threads/translation-for-higurashi-switch-game.513201/page-2#post-9487846

Wow, that's cool! I am very delighted to know that the tool I made helps with this project

Actually, this was exactly what I wanted to do when I started reversing the game =). Just found out that I was not really cut out for the translation part and did not find anybody to do it. If you need/want any help with tooling (like polishing ShinDataUtil up to be of better use for the project) or the game engine (I was looking at it for a good half of a year) - I am willing to help.

[masagrator]

For now I didn't found any issues with compiled scenario in comparison to original file when playing game. Few instances I was thinking they are bugs were also present in original file. I was only slightly surprised that when file was getting smaller with my edits, at some point it went fat and now has 21 MB. Dunno what is the reason for that. Maybe some my more more advanced assembly edits couldn't get optimized properly. :P Well, everything works fine and that's what matters. Only request I have is that when MSGSET has not closed string with ", then it throws array index error and no line number where it occurs. Would be nice to throw error with line number or other thing to identify it where it happens as in other cases where they throw line number.

[DCNick3]

Would be nice to throw error with line number or other thing to identify it where it happens as in case of other cases when they throw line number.

Definitely can do, probably will in near future

(after reading the gbatemp thread) Did you figure out how to compress pictures? I was thinking about the algorithm to do this, but did not come to implementing it. Maybe I can help with that too.

[masagrator]

Yup, I didn't reimplement compression. Just putting decompressed images. For sure implementing compression would significantly reduce rom size. Currently my rom has more than 700 MB. :P

[masagrator]

Now extracting works. Issue can be closed.