Targeting WebAssembly

[PrismaticFlower]

It might be a good idea (albeit a ambitious one) to rewrite targeting webassemly, making the tool work in the browser, and able to be used as a module instead of standalone, which would make for some interesting browser modding tools. Then we could compile on any platform with any browsers supporting wasm (and even remote compilation) with no need for Windows.

I'm interested in doing this, but I'm not sure I can handle it on my own.

Originally posted by @RepComm in https://github.com/SleepKiller/swbf-unmunge/issues/14#issuecomment-568175234

[PrismaticFlower]

It might be more straightforward than you think. The vast majority of the code base should be clean C++17. Although in the recent version I started to use a couple C++2a features. (Designated initializers specifically, which C has anyway so most C++ compilers support them as an extension as well.) So assuming WebAssembly can be targeted using a reasonably compliment compiler and standard library you shouldn't face too many difficulties.

If I remember correctly the one (excluding DirectXTex but more on that in a bit) dependency on the Windows API is in mapped_file.cpp which memory maps a file so swbf-unmunge can operate on it. You could accomplish the same by just using std::ifstream in binary mode and reading the whole file into a std::vector<std::byte>.

As for building as a library, nothing really needs to change to do this. Below is the function in main.cpp that invokes the extraction logic.

void extract_file(const App_options& options, fs::path path) noexcept
{
   try {
      Mapped_file file{path};
      File_saver file_saver{fs::path{path}.replace_extension("") += '/',
                            options.verbose()};

      Ucfb_reader root_reader{file.bytes()};

      if (root_reader.magic_number() != "ucfb"_mn) {
         throw std::runtime_error{"Root chunk is now ucfb as expected."};
      }

      handle_ucfb(static_cast<Ucfb_reader>(root_reader), options, file_saver);
   }
   catch (std::exception& e) {
      synced_cout::print("Error: Exception occured while processing file.\n   File: "s,
                         path.string(), '\n', "   Message: "s, e.what(), '\n');
   }
}

Note that all freestanding functions are thread-safe and "pure" in the sense that the only global state they modify is std::cout/std::cerr (from threadsafe wrappers, see synced_cout.hpp and the filesystem itself. So if you want to use the tool as a library as it were you can simply call extract_file directly or setup your own wrapper around handle_ucfb.

Now for the problems that may come from the dependencies however.

{fmt}

This seams fairly platform agnostic so I assume it'll just work without any fuss.

fx-gltf

Same as above, I don't think it has any dependency on platform specific APIs.

json

Same as above, I don't think it has any dependency on platform specific APIs.

gsl

Pretty sure this is just a C++ 98 library without any platform specific stuff, but I haven't dug too deep into it's source code.

glm

This one can have dependencies on compiler specific stuff or extensions. But I believe it can be told to just keep to clean C++11, so again I assume it'll work.

Threading Building Blocks

This one is more interesting, I have no clue what the status of threads for WebAssembly actually is or what APIs TBB depends on. (I'm pretty sure it's more than just the C++11 threading library since it predates it.)

If it does build and is usable that'd be great. But if it doesn't then you could either wrap the functionality used (tbb::task_group and I think the explode/assemble stuff uses a concurrent vector class from it) into classes that could be stubbed out to simple single threaded versions when compiling for web assembly.

Or if you don't care about merging any changes you make back into master. You could also just remove the multithreading stuff. (Replacing calls to tbb::task_group::run with direct calls to whatever they're running.)

DirectXTex

The main problem, this is where the dependency on Windows comes from. (Also in the texture handling function is an #include of d3d9types.h for D3DFORMAT.

Easiest thing to do would be stub out the texture handlers while you get the rest building and then investigate alternate ways to implement image saving. stb_image_write might be an option.

---

That's all I can think of off the top of my head. Let me know if you want more specific information on anything, if I left anything out or if you want help with anything specific.

[RepComm]

I got a clean template for WebAssembly going and I'm poking around your source to see what I'm getting into. Doesn't look so bad, but I'm not entirely sure where I'm going to start yet.

I think I need to decide what code to keep and what to spam //TODO - replace on.

Threading is no big deal since JS has Workers. I'm thinking of replacing binary reading stuff with a wrapper around js DataView, which has anything a binary reader/writer can need.

Things I'll probably strip first:

• Textures (later feature that I just won't even mess until later)
• FileSystem functionality (just replace with buffer result getting, and leave it up to implementation to use them).

Scrapping textures throws a large portion of work into the future, so that'll definitely happen.

[RepComm]

So the Emscripten SDK does a load of things that I don't need.

It detects system specific things like fs and gl calls and wraps it to web API (like fetch, and WebGL).

It's super cool, but a headache because of the massive overhead and bloat.

I'm following this article to properly grasp how to compile with just clang and simplify a lot of the structure.

Right now the repo is private, but I'll put it public here when I'm compiling a significant portion of modified swbf-unmunge.