[request] standalone version of file_packager.py

[hsdk123]

Currently to package files we need to use file_packager.py - this requires the installation of python.

I also notice that if I just clone this repo and attempt to execute file_packager.py with python installed:

file_packager: warning: config file not found: C:\Users\daego\Desktop\Projects\emscripten\.emscripten.  You can create one by hand or run `emcc --generate-config`
file_packager: error: LLVM_ROOT not set in config (C:\Users\daego\Desktop\Projects\emscripten\.emscripten), and `clang` not found in PATH

signalling even more dependencies, Which aren't even mentioned https://emscripten.org/docs/porting/files/packaging_files.html

[sbc100]

I think it should be possible to make file_packager.py work without an emscripten config yes.

I'm not sure we would consider it high priority at this point since installing emscripten is already something that emscripten users already should know how to do right? Can you explain the use case for wanting to build emscripten file packages without otherwise installing a working version of emscripten?

In any case, if you or anyone else would like to make it more standalone, and add some tests so that doesn't regress, that would be most welcome. Right now that only script I think we consider to be completely standalone is emrun.py.

[hsdk123]

Thanks for the reply @sbc100!

installing emscripten is already something that emscripten users already should know how to do right

I don't think this is necessarily the case - ex. game engines that have UIs that allow users to export their projects to emscripten. The user likely doesn't have any knowledge on emscripten, and having them install emscripten becomes a hurdle.

[sbc100]

Thanks for the reply @sbc100!

installing emscripten is already something that emscripten users already should know how to do right

I don't think this is necessarily the case - ex. game engines that have UIs that allow users to export their projects to emscripten. The user likely doesn't have any knowledge on emscripten, and having them install emscripten becomes a hurdle.

Ah I see, thanks. Do you actually have such a use case on a project you are working on, or is this hypothetical?

[hsdk123]

Ah I see, thanks. Do you actually have such a use case on a project you are working on, or is this hypothetical?

I'm the main developer of a game engine called Light.vn (http://lightvn.net/). We support exporting users' games to the browser by providing a link to emscripten - which is regular cause of user issues.

[sbc100]

Can I ask a little more about your use case? Basically want to check some assumptions:

1. You provide a "deploy to web option" in your engine.
2. The emscripten build of your engine is done ahead of time (by you).
3. Your users then bundle data files, but don't actually do any emscripten compiling or linking?
4. The resulting data is bundled somehow with pre-existing emscripten-built js/wasm?

[hsdk123]

1. You provide a "deploy to web option" in your engine.
2. The emscripten build of your engine is done ahead of time (by you).

Correct

3. Your users then bundle data files, but don't actually do any emscripten compiling or linking

We ask the users to install emscripten and follow the below to create a package.data + loader.js file

https://emscripten.org/docs/getting_started/downloads.html https://emscripten.org/docs/porting/files/packaging_files.html

4. The resulting data is bundled somehow with pre-existing emscripten-built js/wasm?

Which they manually copy paste back in with (1)(2)

[sbc100]

So if file_packager.py was standalone then you could potentially just bundle it with your SDK and then your users would not need emscripten at all?

[hsdk123]

So if file_packager.py was standalone then you could potentially just bundle it with your SDK and then your users would not need emscripten at all?

Correct. I could even just create a script that goes from (1) to (4)

[sbc100]

OK, well you have convinced me its a reasonable idea. If you get any time to work on it that would be great. Otherwise I've maked this bug a Help Wanted in case anyone else has time to look into it.

[hsdk123]

OK, well you have convinced me its a reasonable idea. If you get any time to work on it that would be great. Otherwise I've maked this bug a Help Wanted in case anyone else has time to look into it.

Thank you!

[kripken]

One option for this use case could be a python-to-binary tool like Nuitka --standalone or py2exe. The first step would be making the current python script not depend on anything else in emscripten (which should be simple), then the tool should do the rest.

[sbc100]

I believe that the main task here is actually "making the current python script not depend on anything else in emscripten".. once that is done I think its easy enough to ship it.

Note that "making the current python script not depend on anything else in emscripten" also involves writing tests to verify that works.

[sbc100]

In other words I think the main blocker here is "users shouldn't have to install and configure emscripten" not "user should not have to install python". But I could be wrong.

[kripken]

@hsdk123 can explain the intent, but the top comment is

Currently to package files we need to use file_packager.py - this requires the installation of python.

I also notice that if I just clone this repo and attempt to execute file_packager.py with python installed: (issues because the script has internal deps on emscripten)

So I thought both were issues? Anyhow, yeah, the python issue seems to have tools that help.

Note that "making the current python script not depend on anything else in emscripten" also involves writing tests to verify that works.

Yeah, we have tests for running that file by itself, but we could perhaps make the tests also ensure it doesn't load anything from emscripten while doing so.

[hsdk123]

Thank you all for such enthusiastic responses!

@kripken

So I thought both were issues? Anyhow, yeah, the python issue seems to have tools that help.

Correct, both would be issues - although requiring emscripten seems to be a larger hurdle for average users as that requires more steps.

[ericoporto]

Small question on this. I wrote my own thing that load files in the html js script

https://github.com/adventuregamestudio/ags/blob/a6a912d7ac4a8f3cd8382c07a57ff9e980f55ecf/Emscripten/launcher_index.html#L484

So for my prebuilt engine, there's just the need to output a js file with the list of files (which our Editor makes it): https://github.com/adventuregamestudio/ags/blob/master/Emscripten/my_game_files.js

I didn't knew about this file_packager, what I am losing from not using it? I can't tell exactly what it gives me.

[sbc100]

You should be able to find all the information you need here: https://emscripten.org/docs/porting/files/packaging_files.html

Basically the file packager bundles a bunch of files into a single data file, and then automatically pre-populates the virtual filesystem with them when your program starts. You can of course write the pre-population code yourself as I assume you have done?