Feature request: rom extraction afer compilation

[sheepo99]

It would be nice if rom data extraction could be handled separately from compilation, since this would allow us to build official packages without any rom data. It would also open the possibility for Linux distro agnostic packages such as AppImage and others.

[HengiFettlich]

Yes, we thought about a format reminiscent of DOOM's WAD files.

[Aerocatia]

I'd only request that this be optional as I really like the integrated nature of the current builds.

There is always the option of releasing a patch that patches the original N46 ROM into the package/exe/appimage/whatever.

Lets be real, most people will download the needed ROM's off the internet. This project needs assets form all regions remember, so having the user build a "wad" type thing would need more copyrighted files than just patching one ROM to the release build. Patching this way is at least highly more likely to be properly self-dumped. How many users do you think have all versions of sm64 and a means to dump them?

Adding a whole bunch of code and user requirements so we can pretend this will be used in a more legit way is honestly a waste of time.

[Xenthio]

Danm. Modding Mario 64 as if it was doom would be awesome, essentially making a moddable engine for sm64.

[Tatsh]

Just wanting to see it done like how OpenRCT2 does it where the necessary data from the real game is dealt with by the user. It looks like this would require restructuring the code quite a bit, like needing to put some of the tools onto the user's system so they are always available. Then having the code read the data at runtime rather than directly from the binary.

For Gentoo it's easy to make a compliant ebuild for this in the code's current state: add the ROM filenames to the ebuiild as restricted fetch and ask the user to put a copy of the ROM into the distfiles directory. Then build like normal.

[puddydoo]

It would be very convenient to be able to download binaries without ROM data, but the license for the n64-fast3d-engine renderer prevents us from distributing binaries anyway.

[Emill]

If you have some good idea that works in practice I could maybe reconsider changing the license.

[puddydoo]

Now that there is external data loading, I think there could be a standalone executable that generates a basepack ZIP file from whatever ROM they may have.

[DanTheMan827]

What data is extracted from the ROM when building? are there the means to load all of these types of data from an external source?

[puddydoo]

The files are listed in assets.json and consist of all the textures and audio files of the game. The external data feature in the nightly builds now allows loading all of these from a ZIP.

[fgsfdsfgs]

The sounds technically still need to go through assemble_sound.py and other tools. Can't just load a bunch of aiff files from a zip yet.

[DanTheMan827]

What about dialogue, models, and levels?

Is it possible to build the game without any of these assets included?

The ideal situation would be a binary with nothing included that isn't part of this repo and have the assets loaded from a user provided rom file alongside the binary (as far as easy distribution goes)

I would think it'd be possible to load the rom into ram and give the engine pointers to the assets loaded in ram?

[fgsfdsfgs]

Models, dialogue and levels can't be loaded externally yet, but they are technically part of this repo. It won't be easy loading pointers from a ROM at runtime. Different endianness, pointer sizes, addresses and all that.

[Xenthio]

how will custom textures work in mods if it's all in the file we cannot distribute? can textures also be stored in the executable?

[fgsfdsfgs]

Textures are stored in the executable by default. When EXTERNAL_DATA is enabled, they're loaded from external files (see build/us_pc/res/base.zip), in which case custom textures can be distributed as is.

[robertkirkman]

dominicentek created runtime asset extraction code in C++ that I added a couple things to and then pasted into this repo and it generated an AppImage purely through GitHub Actions without any baserom.us.z64 present and it requires the entire baserom.us.z64 at runtime. And right now it is only compatible with the US version but I sort of think it might be relatively easy to eventually expand to other regions. Is C++ code allowed in this repository or does it having so much C++17 mean it needs to stay in a separate repository? https://github.com/robertkirkman/sm64ex/releases

Edit: it is mostly complete but there are still a few sort of minor bugs with it. I'm going to try to fix it by creating updates and it has been pointed out to me that github actions time gets used up, so it's likely better that I run the builds on my account/repo so that others' time doesn't get expended unless they choose to copy my code, that can be the purpose.

[deadmeu]

dominicentek created runtime asset extraction code in C++ that I added a couple things to and then pasted into this repo and it generated an AppImage purely through GitHub Actions without any baserom.us.z64 present and it requires the entire baserom.us.z64 at runtime. And right now it is only compatible with the US version but I sort of think it might be relatively easy to eventually expand to other regions. Is C++ code allowed in this repository or does it having so much C++17 mean it needs to stay in a separate repository? https://github.com/robertkirkman/sm64ex/releases

Edit: it is mostly complete but there are still a few sort of minor bugs with it. I'm going to try to fix it by creating updates and it has been pointed out to me that github actions time gets used up, so it's likely better that I run the builds on my account/repo so that others' time doesn't get expended unless they choose to copy my code, that can be the purpose.

Would there be much effort in building a Flatpak bundle as well?

[robertkirkman]

Would there be much effort in building a Flatpak bundle as well?

I do not know much about Flatpak, but the Saturn fork of sm64ex has entered the Arch User Repository where it serves as another example of how to distribute a package of this family of sm64ex forks that gain runtime-asset-extraction capabilities. The only issue with that example is that the incorrect handling of package file organization in that AUR package's PKGBUILD makes the package come out malformed when you build it . The only way to fix it is non-trivial so I fixed it and posted my solution in a comment there, so total, there are 2 examples of Arch Linux packaging scripts and 1 example of an AppImage packaging script.