search

larsiusprime / polymod

Atomic modding framework for Haxe
MIT License
160 stars 62 forks source link

Framework-agnostic backend #6

Closed larsiusprime closed 2 years ago

larsiusprime commented 6 years ago

I need to refactor this so that Lime/OpenFL isn't assumed, but is simply a pluggable backend. I should create an abstract interface or something that sits between Polymod and Lime, such that it would be possible in the future to add a Heaps, Kha, etc, backend.

sh-dave commented 6 years ago

Hi there, i'm currently investigating this as well.

Using polymod in this scenario is similar to haxeui V2 and requires the use of 2 libraries: polymod (shared core library) + polymod-XXX (framework specific code)

What i did so far is:

core library

openfl backend

kha backend

Currently i'm trying to figure out how to inject loaded mod assets into kha.Assets.

Let me know what you think!

larsiusprime commented 6 years ago

Hey! This is quite promising. I would love to collaborate with you on this.

I kind of like the idea of keeping everything in one library, and relying on conditional compilation in conjunction with dead code elimination to make sure only the right stuff gets pulled in and any unused framework hooks are ignored. But multiple repos is fine for now as we experiment and work things out.

Basically the three things that need abstracting are:

  1. Code that overrides the framework's default asset loading mechanism
  2. Direct calls to any framework-specific asset system
  3. Framework-specific types for assets (Image, BitmapData, Font, etc)

As an example of a previous effort of mine to make a library framework-agnostic, there's crashdumper: https://github.com/larsiusprime/crashdumper/tree/master/crashdumper/hooks

Crashdumper relies on an IHookPlatform interface to abstract the framework dependency: https://github.com/larsiusprime/crashdumper/blob/master/crashdumper/hooks/IHookPlatform.hx

Here's the OpenFL implementation: https://github.com/larsiusprime/crashdumper/blob/master/crashdumper/hooks/openfl/HookOpenFL.hx

And I have a utility function for optionally auto-detecting the framework: https://github.com/larsiusprime/crashdumper/blob/master/crashdumper/hooks/Util.hx#L30

(Never got around to adding new frameworks to Crashdumper, though it theoretically could be done. Also I freely admit Crashdumper's "framework agnostic" approach might be bad, I wrote it years ago).

Your approach so far looks like more or less what I would do. Nice to see you're thinking on similar lines, makes me feel slightly more confident in my approach.

Let me revisit my three points:

  1. Code that overrides the framework's default asset loading mechanism

One thing to discuss is how Polymod should be used. For instance, it would be much easier to make everything framework agnostic if we just make them replace each and every single framework-specific equivalent "Assets.get()" call in their projects with a PolymodAssets.get() call or whatever. But, that's a huge refactoring pain and one of the appeals in polymod is that you can just drop it into an existing project. In OpenFL this is easy because it has built-in support for overriding your default asset library for just this sort of purpose. We'll come back to this one in a second.

  1. Direct calls to any framework-specific asset system

So if we create PolymodAssets system, it's not too hard to make that framework agnostic. You create a standard interface filled with stubs and then allow an implementation backend to be attached to it depending on what the user needs to use.

In the simplest case, require the user to pass in a list of callbacks for every single getX, getY, getZ call! Then when the user requests a piece of text, or some bytes, or an image, we just defer to those. This might be a good starting point for a "no framework" backend.

For OpenFL, we take the existing implementation and make it the "OpenFLAssetsBackend" or whatever.

This however brings up the next issue:

  1. Framework-specific types for assets (Image, BitmapData, Font, etc)

Polymod requires some knowledge of the asset types. Probably what we're going to have to do here is create our own enumeration list that can be translated to the native asset types of each framework. String and Bytes are universal across haxe projects, and we'll surely have getBytes(assetName) and getString(assetName) as workhorse functions, but we'll need to be able to reformat that to what the native framework expects, and we'll need to be able to do some differing logic depending on the asset type.

It might be best to start with 2. and 3. and leave 1. as the final step. Once we know how to provide a generic PolymodAssets interface for fetching stuff, and make sure that can line up with any asset system, then we can look into how to use each framework's capabilities to seamlessly allow overriding of the default global asset fetch system, if that is indeed possible. If the framework doesn't support it, the user will have to call PolymodAssets directly probably.

larsiusprime commented 6 years ago

Probably the best first step is also to just get a good accounting for the differences in all the various framework's asset systems, so we can start with a plan that doesn't paint us into a corner. Could you tell me all about Kha's asset system, for instance?

sh-dave commented 6 years ago

1) I'm very much in favor of using the framework specific asset loading on the users side. The PolymodAssets wrapper class i created isn't supposed to be used by them, but only as a core-libary to framework-specific thing. This would be the place where the framework specific asset injection happens. Thatswhy it's implemented in each of the backend repos. So user code would still use the framework's asset management:

3) Why does polymod need to know the type of asset, for the text-merging (i haven't really looked too deep into everything the lib has to offer yet)?

Kha asset handling

As for asset handling in Kha, it's similar to OpenFL, except you don't have the special TEXT type, but only Blobs (= BINARY). The actual format of text assets is something you handle manually in your game/app. So you have:

And all assets are converted during preprocessing into platform specific formats. Take for example sounds. Source files in Kha are always WAV. When you build for html5, they get converted to OGG, MP3, MP4. Additional info is in the wiki

To access asset you have a number of ways:

As for actual interfaces in code, IMO they only make sense for code that executes differently during runtime. Say a generic Resource interface that's implemented by ImageResource, SoundResource, ... Designing the whole library via interfaces is an unnecessary limitation, but my opinion on this isn't too strong.

larsiusprime commented 6 years ago

Thanks for this information!

The main reason that Polymod needs to know about asset typing really just comes down to TEXT vs NOT-TEXT. For all binary files there's basically only one thing Polymod can do with it -- let a mod replace the asset wholesale.

For text based assets, Polymod has three different things it can do with the text:

  1. Replace it wholesale, just like with a binary asset
  2. Allow one or more mods to successively append new lines of text to the end of the file
  3. Allow one or more mods to successively inject new data segments into specified locations in the data structure defined by the text

2 & 3 require that Polymod itself be able to not only load the files, but also detect their format (TSV vs CSV vs XML vs JSON, etc), parse them, apply all the intended changes from one or more active mods, and then return the final composed version.

This does require some information from the user. Right now, the file formats are all hard-coded (if it sees TSV or CSV or XML extension it treats the file as that format), but I plan on making this something the user can define. Furthermore, there's already a very basic structure for the user to provide their own parsing logic (CSV is a notoriously ambiguous format, so I've provided a very basic way to specify how Polymod should treat a project's CSV files).

larsiusprime commented 6 years ago

Another reason that Polymod might need to know about asset formats is to properly handle caching. OpenFL's asset management by default caches what you load, so the second time you ask for asset xyz it returns the cached version rather than fetching it from disk. IIRC, the cache might be organized around asset types, so it might cause subtle breaks in behavior if you e.g. simply fetch audio files as naked bytes and feed that directly into an audio object -- in OpenFL at least, that might lead to the cache being skipped and next time you ask for that audio asset it could load it from disk all over again. Will have to look into this.

larsiusprime commented 6 years ago

Yeah, confirmed: https://github.com/openfl/lime/blob/78e99bf1d92266a06605e32147c9c6ba3a6a03dc/src/lime/utils/AssetCache.hx#L20

Lime's default asset cache behavior is to have three separate typed caches for Images, Audio Buffers, and Fonts -- storing the fully parsed/loaded object in cache rather than just the raw bytes on disk.

In Lime, the act of fetching the asset itself is also when the cache gets set: https://github.com/openfl/lime/blob/0c96d222869be5251b85ba5c5568da473dcedc4c/src/lime/utils/Assets.hx#L152

And fetching an asset with the generic "BINARY" type signals to skip the cache: https://github.com/openfl/lime/blob/0c96d222869be5251b85ba5c5568da473dcedc4c/src/lime/utils/Assets.hx#L94

So for frameworks that have this sort of caching behavior, care will have to be taken that overriding it with Polymod's implementation will not cause unintended behavior.

larsiusprime commented 6 years ago

Okay! So I'm done refactoring. The OpenFL backend works like it did before, but now it's separated by a layer of abstraction, and you can slot in a new backend there. I've put in stubs for Kha, Heaps, NME, and LIME.

Current plan:

In the end I went with the "keep everything in one repo" approach, and I found that the Backend interface doesn't really need to do much other than expose a few very simple functions. The bulk of the actual work is done entirely in the backend code, where you override your framework's default asset library.

sh-dave commented 6 years ago

Yup will do, I'll clean up my sample code (port of the openfl_hscript one) and contribute that as well. Got the scripts working yesterday, and need to work on unloading mods (or the "default" asset library) now. Then i'll take a closer look on the new branch.

larsiusprime commented 6 years ago

*NME proved more challenging than I thought, it'll require assistance from someone like @hughsando to help explain to me how the asset system is structured and how to override it.

hughsando commented 6 years ago

What is it you are trying to do? You can set asset data directly to the Assets.info map if this is what you are thinking.

larsiusprime commented 6 years ago

@hughsando:

Take a look here: https://github.com/larsiusprime/polymod/blob/master/polymod/backends/OpenFLBackend.hx

What I do in OpenFL is create a custom AssetLibrary that I replace the default one with, that is the first stop for receiving any Assets.get<Something>(id) calls.

Then, it does this:

Basically I need to know what the proper API do this in NME is. I got a little confused trying to sort out asset factories which seem to operate on a per-assettype basis. Does Assets.info map let me set one global custom "asset library"?

larsiusprime commented 6 years ago

Just took a look at AssetInfo, assuming I didn't make a mistake: https://github.com/haxenme/nme/blob/master/src/nme/AssetInfo.hx

So you've got a map of strings (presumably the asset id) associated with AssetInfo metadata. What I'm trying to do is find a way to have my overridden library short-circuit the normal asset fetching from disk if a modded file is present, and just return the modded version directly from my system before it falls back to NME's default behavior.

From the looks of it, I might be able to do this by extending the AssetInfo class and abusing the getCache() function?

https://github.com/haxenme/nme/blob/4a910b790e6c78f9a04251f993cfcd8579a6695d/src/nme/Assets.hx#L319

hughsando commented 6 years ago

So if you know the list of mod files at some time (say, startup), the you can call something like "addModFIles()" from the main routine, which would do something like:

for( mod in modFiles())
    Assets.info.set(mod.id, new AssetInto( .... ) )

If you want it to me more dynamic (ie, after startup), will need to mod the 'Assets.getInfo' function to call out, say by adding an array of "AssetFinders" which would be String->AssetInfo functions

larsiusprime commented 6 years ago

@hughsando -- Startup is fine, that's the recommended use case.

If I set my own asset info, that's a mapping of an asset id to a new filename on disk, right? I take it that in this situation NME is still ultimately responsible for making the file system calls that actually fetch the asset? That will probably work for my purposes; though it will mean NME is limited to supporting SYS targets (unless there's a way to point NME to a custom virtual filesystem)

hughsando commented 6 years ago

I think sys is fine - win/linux/mac/android are all good. IOS - I guess you need to read some common app/documents directory, but should be fine to the extent that IOS allows you to mod like this. JS - you could use urls. Cross-site scripting may be an issue depending on who is doing the modding, but I think you should be covered for 99% of the use cases.

larsiusprime commented 6 years ago

@hughsando -- the one big issue is text files. With text, we don't just support wholesale file replacement, but also allowing mods to append new lines, or inject payloads into target sections of e.g. xml. This requires polymod to be able to fetch the asset and return the bytes for it --- which I think with NME is currently impossible? I can point to a new file, but nme does the loading. We could do it with temp files and path rewrites maybe, but that feels messy. Am I missing something?

larsiusprime commented 6 years ago

@hughsando -- okay, just got file replacement working! Works pretty much perfectly, very easy to do, exactly as you described.

My main focus for now is desktop sys target as HTML5 introduces all sorts of security questions, and mobile isn't exactly a hotbed of modding activity as is. If I can get append/merge logic working for text I'll consider that just fine for my purposes.

larsiusprime commented 6 years ago

@hughsando -- okay, I was able to abuse the cache functionality to get text appending & merging working!

for (key in nme.Assets.info.keys())
{
    var info = nme.Assets.info.get(key);
    if(info.type == TEXT){
        var origText = PolymodAssets.getText(key);
        var newText = polymodLibrary.mergeAndAppendText(key, origText);
        if(origText != newText)
        {
            var byteArray = nme.utils.ByteArray.fromBytes(Bytes.ofString(newText));
            info.setCache(byteArray, true);
            info.isResource = false;
        }
    }
}

Basically what I do is for every piece of text in the asset list, see if I have any append/merge instructions and resolve them. If the resulting text creates a difference, then store that difference in the cache for that particular asset, and tell the system to ignore its haxe.Resource value.

The big downside to this method is that it's effectively preloading every single text file in your entire asset library at startup and then caching the result and keeping it in active memory for the life of the app. Text files usually aren't super huge, so for a great many apps I doubt this will be an issue, but it could be a major "gotcha" and I'd prefer to find a better way if it exists, especially because this "cache hack" is exclusive to my NME implementation.

My ideal solution is what I'm doing in the other backends:

Is this possible? Is there some API functionality in NME that I'm overlooking? Maybe asset factories?

hughsando commented 6 years ago

You can set a custom way of getting the data by using the "byteFactory" in the nme.Assets, which seems to only get triggered if the asset is not a resource. So maybe if the bytes are a resource (info.isResource) use your existing method. The bytes will already reside in memory, so I think this limits overhead. The non-resource path does this:

          if (byteFactory.exists(filename))
               data = byteFactory.get(filename)();

So, for the polymod overrides (or all of them), iterate over the assets and set

   nme.Assets.byteFactory.set( info.path, Polymod.readFile );
    // or, if you need the id:
   nme.Assets.byteFactory.set( info.path, function(filename) return Polymod.readFile(info.id, filename)  ) ;

The readFile function should essentially do: ByteArray.readFile(filename); and make any mods from there.

larsiusprime commented 6 years ago

@hughsando Wonderful, Hugh, that was exactly the ticket!

Here's the final code I wound up using. Seems to work perfectly!

for (key in nme.Assets.info.keys())
{
    var info = nme.Assets.info.get(key);
    if(info.type == TEXT){
        if(info.isResource)
        {
            var origText = PolymodAssets.getText(key);     //falls through to normal Assets.getText(key);
            var newText = polymodLibrary.mergeAndAppendText(key, origText);
            if(origText != newText)
            {
                var byteArray = nme.utils.ByteArray.fromBytes(Bytes.ofString(newText));
                info.setCache(byteArray, true);
                info.isResource = false;
            }
        }
        else
        {
            nme.Assets.byteFactory.set( info.path, function(){
                var bytes = PolymodFileSystem.getFileBytes(key);     //read bytes from sys filesystem (but in the future, PolymodFileSystem can have a virtual file system slotted in instead)
                var origText = Std.string(bytes);
                var newText = polymodLibrary.mergeAndAppendText(key, origText);
                if(origText != newText)
                {
                    return nme.utils.ByteArray.fromBytes(Bytes.ofString(newText));
                }
                return nme.utils.ByteArray.fromBytes(Bytes.ofString(origText));
            });
        }
    }
}
hughsando commented 6 years ago

Love it. Glad you did not need to change the nme code.

larsiusprime commented 6 years ago

@hughsando -- Indeed :) I suspected NME had some sort of facility like this, just needed someone more familiar than me to point out where it was hiding.

larsiusprime commented 6 years ago

Just committed the Vanilla Lime backend. That just leaves Kha and CastleDB.

EliteMasterEric commented 2 years ago

The actual functionality for interchangable backends is fully implemented, and backends for most frameworks (including HEAPS, Lime, NME, and OpenFL) have already been provided, and functionality to allow for custom backends is in place too.

I think it is more fitting for the remnants of this issue to be split into #57 and #58 for CastleDB and Kha respectively, and close this one.