search

Xaymar / Hellextractor

A simple tool to extract things from Helldivers 2 for your 3D printing needs.
BSD 3-Clause "New" or "Revised" License
32 stars 5 forks source link

stingray, mapped_file: Various fixes #30

Closed lat9nq closed 4 months ago

lat9nq commented 5 months ago

stingray and stingray_unit had issues that prevented me from compiling. Once I got through that, there was an issue with mmap that caused it to not work, then the program misread the error as a success and ran into a crash. Once I fixed that, I enabled submodules so I could make sense of the files I was getting. This caused all of the open calls to error with 24, AKA EMFILE 24 Too many open files, but I had to put the errno in the error message to find that out.

Fixed the warning in stingray that prevented clang from compiling it with a static cast. Also cleaned up an include there that my IDE told me about. stingray_unit needed to include vector so it's there now. Added MAP_PRIVATE to the mmap call, so now it functions as it should. Currently I'm running ulimit -S -n 65535 before I run the program so that it the open calls work -- not sure if there's a better way to fix this in code.

At this point, I can look at the game's music files, but vgmstream can't read it. I think this is out of scope of this project, though, lol. (Help appreciated)

xypwn commented 5 months ago

Vgmstream definitely can read the wem audio files. The bnk files are "encrypted" in 2 header bytes with an XOR key. What did you try that didn't work? Btw, feel free to look at my implementation. I basically rewrote/copied the code of various projects (ww2ogg, bnkextract, vgmstream) and added some of the HD2 specific stuff like the XOR decryption.

lat9nq commented 5 months ago

Thanks, @xypwn! I was trying to open the files simply by running e.g. vgmstream-cli 193288922.wwise_stream but I would get an error. As for your filediver program, it would try to find the Steam library in a weird directory by default (~/.local/share/Steam/steamapps/libraryfolders.vdf is common, but filediver omits the Steam portion of the directory). I manually specified the game after that, but none of the files would extract (e.g. filediver -g ~/.local/share/Steam/steamapps/common/Helldivers\ 2/ -c "audio:format=source" -i 'content/audio/*.wise_stream' succeeds with INFO Extracted 0/0 matching files).

xypwn commented 5 months ago

Thanks for the feedback @lat9nq . TBH I haven't actually tested filediver on Linux yet, as I run HD 2 on Windows. It would be great if you could open an issue on filediver's GitHub so we can work on making it function on Linux!

Regarding vgmstream, I read the source code and it explicitly requires wwise stream files to have the ".wem" extension in order to be recognized. Then it should work.

Xaymar commented 4 months ago

not sure if there's a better way to fix this in code.

It could be changed to only open files when needed for indexing and extracting. While this would slow down Windows significantly, due to the code using a "hidden" mechanic of FILE_SHARE_READ and FILE_FLAG_RANDOM_ACCESS, on *nix it would make the application run without adjusting ulimit. As far as I know, there's no current reason to keep ulimit low anyway.

I think prlimit does what we want in this case?