[Request] Re-enable support for Asset Bundles

[IrrSoft]

This is a simple feature request that I had no idea where to post, so I hope here it is OK.

I've been using Xenko for a few months now and I'm really loving its current workflow. It has adapted quite well to mostly everything my project has needed except for one thing : the now obsolete (at least according to the documentation) Asset Bundles, since at this moment there is no way to build them nor load them. This is preventing me from finishing my project and releasing it since it would make it unnecessarily cumbersome to release expansion packs and updates as I would need to rebuild and re-upload the whole thing and have users download the full package over and over again, which is not the best solution.

I hope that at some point in the roadmap Asset Bundles come back since, at least in my own experience, Xenko is very usable already for small and mid-size indie projects except for this feature that has sadly been disabled between versions.

Thanks for reading and for such a great tool, I'm very excited to keep following it, using it and to see where it goes!

[Eideren]

As a workaround for now you could build patches through binary difference, I used bsdiff for a couple of projects and it's pretty reliable.

[IrrSoft]

As a workaround for now you could build patches through binary difference, I used bsdiff for a couple of projects and it's pretty reliable.

Thanks! That is indeed a good workaround for patches and I will make sure to look into it to integrate it to my project. Hopefully the bundles come back for expansion packs/DLC/extensions workflow as well :)

[xen2]

@IrrSoft I don't remember why it was obsoleted (probably lack of UX/testing). If easy to do for you, feel free to give it a try (using doc) and let me know how it goes for you. Otherwise @Eideren approach is a good workaround.

[IrrSoft]

@IrrSoft I don't remember why it was obsoleted. If easy to do for you, feel free to give it a try (using doc) and let me know how it goes for you. Otherwise @Eideren approach is a good workaround.

Hi there, thank you for your reply! I have given it a try following the instructions carefully ( defining the bundles on the xkpkg file ) and it does detect the assets assigned to the bundle and tries to build them, however it throws an error saying that the asset could not be found. It gives me the exact file name of the asset I added to the bundle but it also says that the asset could not be found and sends me a build error, so I am not sure what is happening behind the scenes. It finds all assets with the tag to add them to the bundle, but can't link the assets themselves while building.

EDIT : It sends me the error in the following format : Can not find 'correct/relative/path/to/MyAsset' for 'MyCustomBundle' so it is detecting the assets with the given tag, it just doesn't find them for some reason when building the asset bundles themselves. I hope this helps somehow :)

[DW01]

The engine generates the default bundle when you create a project, so it definitely still works. Odd that custom bundles don't generate correctly. The error seems to suggest that invalid path is hardcoded for some weird reason?

[IrrSoft]

The engine generates the default bundle when you create a project, so it definitely still works. Odd that custom bundles don't generate correctly. The error seems to suggest that invalid path is hardcoded for some weird reason?

I have no idea. The workflow for custom bundles is (according to the documentation) to add the tags Xenko will use to track which assets to put on which bundle. Once this is done and I hit build, Xenko displays errors saying that assets "this" and "that" could not be found (this and that being the correctly tagged assets) all while displaying the correct paths to said assets which means Xenko is trying to build the custom bundle and correctly identifying the assets but for some odd reason not finding them by path,, so I have no clue as to what is preventing the custom bundles from building. Everything else on the default/root bundle works perfectly indeed

[xen2]

This codepath has been unused/untested for a while, hopefully a quick debugging session should sort it all out. Let me know if you need help there.

[IrrSoft]

This codepath has been unused/untested for a while, hopefully a quick debugging session should sort it all out. Let me know if you need help there.

Thanks ! Some help with it (to at least point me in the right direction) would really be appreciated since ATM I have been using the Xenko binaries only and I noticed the issue almost 2 months ago when trying to follow the example in the documentation. I'll try to do some debugging on my own as well. Again, thanks for the fast replies!

[xen2]

OK, let me have a quick look then

[xen2]

@IrrSoft I might have identified the issue. As a workaround, can you try to mark those assets as Root Assets?

[xen2]

Seems like behavior need to be decided to properly fix the bug: right now we have this issue because we don't build unreferenced assets but we still try to include them in the bundle anyway

(1) Include all assets that matches the bundle even if not referenced (either as root or indirectly) -- i.e. being part of a bundle mark them as root asset implicitely Pros: easier to use since those assets will most of the time be unused Cons: going against what is done for default bundle, and difficult to enable/disable some assets

(2) Include assets that matches the bundle only if they are referenced (either as root or indirectly) -- i.e. we don't try to bundle anything that's not referenced

Basically it boils down to: is the bundle packer a simple filter, or can it creates implicit references as well?

[IrrSoft]

@IrrSoft I might have identified the issue. As a workaround, can you try to mark those assets as Root Assets?

Marking them as root assets does indeed allow me to build the project, but it seems that unfortunately the only bundle built is the default one unless I am not searching the bundles in the right place :

Hope this helps, and good to know there might be a way to solve it!

Edit : IMO option 2 might be the best one since it would give a bit more control over what gets included on each bundle.

[xen2]

Strange, I see the new bundle file. However it seems this doesn't completely fix the issue anyway as I see this bundle taking only 1kB (it should contain a texture).

[IrrSoft]

I see. For me, without marking the asset as a root asset, I see the same error from before "Asset not found at " followed by the asset's path (minus the extension) and "for MyAssetBundleName", while marking it as root there's no error, but no additional bundle either.

I'm using version 3.0.0.4 if that helps!

[xen2]

OK, my bundle staying at 1kB seems to be because the streaming system (used for textures) is saving textures as string rather than explicit references, so they are not collected. This will need some investigation/changes and might take little bit of time to fix (bundles won't work properly without this).

Please use bsdiff as recommended by @Eideren for now, and hopefully they will be back soon.

Let us know if you have any recommendations as whether to follow (1) or (2) as described earlier.

[IrrSoft]

Thanks for your help and for following it so quick, it's really good to hear it's being looked into. I'll check well later today the implementation we're doing for our project to give any feedback/recommendation that might be helpful regarding those two possible solutions and use @Eideren 's suggestion for now to include at least the patch system. Thanks again!

Edit : Personally, if option 2 has less conflicts with the way the default bundle is built I would prefer that, since I don't mind having to manually mark down those assets as root assets. I prefer this added level of control and consistency over what gets included, even if it means some additional manual work to mark assets as root or referenced.

[xen2]

@IrrSoft I could find a fix for the second issue (texture streaming). I will fix the original issue after thinking little bit more whether (1) or (2) is better -- this one should be quick to fix once this is decided.

[IrrSoft]

Thanks, it's great to hear that, and to see that it seems like the issue can be solved sooner :)

[Eideren]

@xen2 I haven't seen much changes related to asset bundles, so this is still an issue right ? Is it in a state where we could offload this to the community while you work on other things ?