EsotericSoftware / spine-runtimes

2D skeletal animation runtimes for Spine.
http://esotericsoftware.com/
Other
4.43k stars 2.92k forks source link

[unity] Delayed on-demand loading of Atlas assets #1890

Open HaraldCsaszar opened 3 years ago

HaraldCsaszar commented 3 years ago

Mentioned on the forum here: http://esotericsoftware.com/forum/How-to-control-texture-memory-usage-for-skeleton-with-skins-13240 http://esotericsoftware.com/forum/I-tried-to-export-with-command-line-but-not-file-is-created-14258?start=25 Also requested on this forum thread: http://esotericsoftware.com/forum/Memory-management-of-character-with-many-outfits-15867

In Unity all referenced assets are automatically loaded, so there should be a comfortable way provided that allows to:

  1. Assign multiple atlas assets at the SkeletonDataAsset in the Unity editor and keep them as-is upon reimport, even when not all attachments are found. A preferences parameter shall allow disabling any warnings when not all attachments are found.
  2. Load assigned atlas assets (textures) on demand when attachments are required of this atlas. Could be managed by attachment name prefix like green/ or tier4/. There should be a checkbox Load on-demand provided in the Inspector to enable on-demand loading.
  3. Unloading of no longer used atlas textures shall be triggerable via code. Auto unload shall be configurable per atlas asset as an Inspector checkbox.

Implementation notes: ad 1:

ad 2:

jacattrongnlh commented 1 year ago

Is there any plan to implement this soon? Our game is having memory issue related to this, it could really use this feature

HaraldCsaszar commented 1 year ago

@jacattrongnlh If you need a solution soon, we would highly recommend implementing a solution on your own in the meantime.

HaraldCsaszar commented 1 year ago

First of all, sorry for the long delay! A first experimental version of an on-demand Addressables texture loader has now been implemented.

You can try it out using the following steps:

1) Downloading the latest spine-unity 4.1 unitypackage from the download page or updating your 4.1 spine-unity runtime via git to the latest revision. This runtime version provides basic integration of the abstract base class OnDemandTextureLoader from which on-demand loaders derive from.

2) Install the newly added UPM packages com.esotericsoftware.spine.on-demand-loading and com.esotericsoftware.spine.addressables. You can install them via the Unity Package Manager if you've installed the spine-unity runtime via the Package Manager as well, using the normal Add package from git URL .. button using the following git URLs:

https://github.com/EsotericSoftware/spine-runtimes.git?path=spine-unity/Modules/com.esotericsoftware.spine.on-demand-loading#4.1 https://github.com/EsotericSoftware/spine-runtimes.git?path=spine-unity/Modules/com.esotericsoftware.spine.addressables#4.1

If you've installed the spine-unity runtime via unitypackage, you need to download the two packages above and before adding them to your project remove the line "com.esotericsoftware.spine.spine-unity": "4.1.21" from both package.json files (removing the spine-unity UPM package dependency). More comfortable official zip packages will be released soon, this only serves as a first test package to don't keep you waiting longer than necessary.

For usage of the Addressables on demand loader package, please see the readme.md file in com.esotericsoftware.spine.addressables/Documentation.

Please share your thoughts on the forum, especially if you encounter any problems or have ideas for improvement. Hope you like it! :)

HaraldCsaszar commented 1 year ago

On-demand (Addressables) texture loading now supports SkeletonGraphic. OnDemandTextureLoader received an added method RequestLoadTexture and delegate TextureRequested. Fixed missing loader asset name suffix.

New spine-unity 4.1 and 4.2-beta unitypackages are available for download: https://esotericsoftware.com/spine-unity-download

You can update and try out the two UPM packages as described in the above comment.

As always, please let us know if you encounter any issues or have any ideas for improvement. :)

Gamer-XP commented 1 year ago

I needed something like this for the project, and decided to test it. It kinda does what I wanted in theory, but:

  1. When you start and stop the game - it does not revert materials back to initial state, resulting in them referencing unloaded addressables (now null textures).
  2. If you start the game again in this state - materials become pink and won't get fixed till you manually remove and re-add all references in spine assets
  3. Or, in some cases, materials may start referencing hi-res original texture on game stop. In such state building addressables will generate duplicate of hi-res texture, defeating the purpose of having addressables in the first place.
  4. Pressing Regenerate in AddressableTextureReference throws an error. It creates texture though.
HaraldCsaszar commented 1 year ago

@Gamer-XP Sorry to hear you're having troubles. Unfortunately we could not reproduce any of this behaviour, testing any Configurable Enter Play Mode options as well. Could you please send us a minimal Unity project which still shows this issue? You can send it as a zip package to contact@esotericsoftware.com, briefly mentioning this issue ticket URL so that we know the context.

Please also note that activating the placeholder textures via Testing - Assign Placeholders inside the Editor and entering and exiting play-mode is for in-Editor preview purposes only. Placeholder textures need not be assigned manually, as these are automatically assigned via a pre-build step (and reset in the post-build step).

Gamer-XP commented 1 year ago

Maybe I'm misunderstanding something. Actually, I planned on using those placeholders as temporary graphics behore Unity loads full-res assets from the server. So, I assume player may actually see them for a while if net or device is slow. That's why I find it weird that those placeholders are quite buggy.

And yes, after testing, this issue happens only if I used Assign Placeholders before starting the game - it won't revert materials back to placeholders after game end, instead I get null textures there.

Also, didn't know about pre-build step. Still, about that. I'm not 100% sure yet, but it doesn't seem to work with building addressables themselves at least. I tried doing analyze for duplicates, and seems like it still includes high-res texture in both addressable bundle and main project if you have spine object in the scene at least. This is the reason I tried to use low-res texture for the material as default - because I wanted to remove reference to the high-res one to prevent this duplication.

I'm using Unity 2022.3.30f, Runtime 4.2-beta btw

HaraldCsaszar commented 1 year ago

@Gamer-XP The placeholder assets are assigned automatically to avoid having blurry low-res textures during development, or having to replace them all manually before building, either of which would be terrible to work with.

That's why I find it weird that those placeholders are quite buggy. And yes, after testing, this issue happens only if I used Assign Placeholders before starting the game - it won't revert materials back to placeholders after game end, instead I get null textures there.

If reset after exiting play mode does not work as expected, please send us a minimal reproduction Unity project as described above, so that we can fix the issue.

. I'm not 100% sure yet, but it doesn't seem to work with building addressables themselves at least. I tried doing analyze for duplicates, and seems like it still includes high-res texture in both addressable bundle and main project if you have spine object in the scene at least.

Did you assign the normal high-res textures at an addressable bundle? Or did you leave it at Addressable unchecked (which would be wrong)?

This is the reason I tried to use low-res texture for the material as default - because I wanted to remove reference to the high-res one to prevent this duplication.

Thanks for the info, sorry to hear. We would really like to fix the issue if it behaves like that in your project. As mentioned above, it would help us a lot if you could send us a minimal Unity reproduction project.

HaraldCsaszar commented 1 year ago

I'm using Unity 2022.3.30f, Runtime 4.2-beta btw

@Gamer-XP BTW: Unity version 2022.3.30f does not exist yet, 2022.3 is at 2022.3.11f currently. Did you mean 2021.3.30f?

Gamer-XP commented 1 year ago

Yep, sorry, a typo. 2021.3.30f. I'll try making an example project later. A bit busy for now.

HaraldCsaszar commented 1 year ago

@Gamer-XP Great, thanks.

Gamer-XP commented 1 year ago

Here you are: SpineAddressableIssues.zip So, how to reproduce:

  1. Windows -> Asset Management -> Addressables -> Analyze
  2. Make suere there is full-res character in the scene
  3. Press Analyze selected rules with all rules selected
  4. Confirms there characters' texture is listed in Check Scene to Addressable Duplicate Dependencies - this means this texture is both in addressable bundle and in the main project now bacuse it's refrenced by the character's material
  5. Replace texture in the character to the low-res version
  6. Analyze again
  7. No errors now, since low-res texture is not an addressable asset
HaraldCsaszar commented 1 year ago

@Gamer-XP Thanks for sending the reproduction project. Please note that the Addressables - Analyse functionality will report the original textures to be referenced twice by the scene and the addressable group, since the original full-res texture will be replaced as a pre-build step, which the Analyse run does not trigger. So thanks for bringing this to our attention, we will have a look if we can improve the situation to make the analyse run more useful and not report false-positives in this regard.

Regarding the other more important issues you reported above: Unfortunately we could reproduce neither of the issues, everything worked without any issues using your project. Since your repro-project was created with version 2022.3.1f1, we used the latest 2022.3.11f1 version (note the .11 vs .1) for testing. Are you sure that you've been using the latest Spine On-Demand Loading Extensions and Spine Addressables Extensions UPM packages, version 4.1.0-preview.2?

Gamer-XP commented 1 year ago

Weird. It works fine in the example project for me too, but is bugged in the actual one. I'll try checking what's going on here.

HaraldCsaszar commented 1 year ago

@Gamer-XP Thanks for confirming. Please have a check whether the version in the original project is really 4.1.0-preview.2 at both packages, and not 4.1.0-preview.1 in the Unity Package Manager window.

HaraldCsaszar commented 1 year ago

Fixed an issue with editor not resetting on-demand loaded textures after exiting play mode, see commit 7eea8ce8.

Gamer-XP commented 1 year ago

Thanks. I'll check it when I get time. We've solved texture loading in different way for now (by swapping texture in materials, while textures are automatically generated when building addressables), and I didn't have time to check on-demand loading yet.

liuxiaotian commented 11 months ago

This commit looks like it increases the build time. With this commit included, our project's build time jumped from 10 minutes to 20. Reverting this commit could get our build time back to normal.

HaraldCsaszar commented 11 months ago

@liuxiaotian Thanks for reporting and sorry for the troubles. I assume you mean the last commit above, https://github.com/EsotericSoftware/spine-runtimes/commit/7eea8ce8b31650c02451f6da7dc953b1526494c2, right?

We will have a look at it.

HaraldCsaszar commented 11 months ago

@liuxiaotian Unfortunately we could not reproduce the issue. When starting or ending a build, this callback of changing play mode should not be called at all. Even if it were called, only the assets of type OnDemandTextureLoader would be processed.

Which exact commit did you revert? Which Unity Editor version are you using? Could you please describe in more detail what your setup is? Are you having many Assets of type OnDemandTextureLoader in your project, or do you have any at all?

Could you add a Debug.Log() statement to GenericOnDemandTextureLoaderInspector.AssignTargetTexturesAtAllLoaders() which shows whether this method is called when starting or finishing a build in your project?

Or even better: could you perhaps create a minimal Unity project which still shows this issue? If so, you can send it to contact@esotericsoftware.com, briefly mentioning this issue ticket so that we know the context.

liuxiaotian commented 11 months ago

Sorry for the late reply. I'm using Unity 2020.3.41f1, the commit is 76e8538. SpineBuildProcessor.PreprocessBuild() takes around 70 seconds when executed directly, it takes longer during building. Here is my test code.

[MenuItem("Test/Spine Preprocess Build")]
static void Menu()
{
    var sw = new Stopwatch();
    sw.Start();
    SpineBuildProcessor.PreprocessBuild();
    sw.Stop();
    UnityEngine.Debug.LogError($"Total time: {sw.ElapsedMilliseconds}ms");
}

profile

sandolkakos commented 10 months ago

hi @HaraldCsaszar, first of all thanks a lot for the initiative of On-demand and Addressables modules. We have a character with 70+ skins and all the textures is getting loaded into the Memory, but now with these modules we were able to solve this issue.

It has almost 1 month since we started to investigate how to use the modules, and came up with different results:

After days of investigation, today I found the reason for the 3 failures above... The Materials were not set with the low resolution placeholders in the Addressable bundles.

By default, the Addressables Build process starts before the App Build process. That means this line below is executed only after the Addressables already got completely finished with all the Materials using the original High resolution textures.

In our project, the build pipeline has already a system in place where I can trigger the OnDemandTextureLoader.AssignPlaceholderTextures(); before the Addressables Build starts, but it would be nice if you could also add that automation to the plugin itself, so that people will not run into the same situation we were.

HaraldCsaszar commented 10 months ago

@sandolkakos Thanks very much for the feedback and for sharing your insights, these are very valid points! Sorry you were having troubles. We will have a look at what we can do to automatically cover these cases as well without adding any overhead.

HaraldCsaszar commented 10 months ago

@sandolkakos Thanks again for reporting your issue. The above commit for spine-unity (the core-package) on the 4.2-beta branch fixes pre-build-step order in Unity 2021.2, where Addressables and AsserBundles are built before the SpineBuildProcessor.PreprocessBuild hook is called.

Details: This was due to BuildPlayerProcessor subclasses callback hooks being called before IPreprocessBuildWithReport subclasses regardless of callbackOrder. Now SpineBuildPreprocessor is derived from BuildPlayerProcessor on Unity 2021.2+.

Since this could be a breaking change for existing build pipelines on 4.1, this change has been pushed to 4.2-beta only. A new 4.2-beta spine-unity unitypackage is available for download: https://esotericsoftware.com/spine-unity-download

The above change should automatically fix the order of pre-processing on-demand-loaded textures before building Addressables via Build Addressables on Player Build. Unfortunately we haven't yet found a way to automatically pre-process single AtlasAssets when building only Addressables via the Addressable Groups window. If anyone knows a solution to this, please do let us know!

Gamer-XP commented 10 months ago

For building addressable only, the solution I've made is custom shemes (AddressableAssetGroupSchema) + custom addressable builder script (BuildScriptPackedMode). I don't think there is any way to modify existing build pipeline for addressables. At least, I couldn't find anything.

sandolkakos commented 10 months ago

Thanks a lot for that fix, @HaraldCsaszar. I'm always happy to help. We are not gonna update to 4.2 yet since we were able to handle that issue in our build pipeline, but I'm sure people will find it super useful and, in the future, when we update to 4.2 we will be able to remove our workaround. 👍🏽

HaraldCsaszar commented 10 months ago

@sandolkakos Glad it helps, thanks for your reply! If you find anything how we can further improve build automation, don't hesitate to let us know.

sandolkakos commented 10 months ago

In our project, we use Github actions to build the App, and before starting the build we can prepare a lot of stuff on the project. For local builds in the developers machine, we have created a different script to interrupt the build when you click the Build button at Build Settings window in order to execute our preparations and then continue with the build process. Something like this:

[InitializeOnLoad]
public static class LocalBuildScript
{
    static LocalBuildScript()
    {
        if (Application.isBatchMode)
        {
            // This script should not be used in batch mode.
            return;
        }

        BuildPlayerWindow.RegisterBuildPlayerHandler(BuildPlayerConfirmation);
    }

    private static void BuildPlayerConfirmation(BuildPlayerOptions options)
    {
        // Implement the pre-build preparations here

        // Continue the build
        bool success = BuildPipeline.BuildPlayer(options);
    }
}
sandolkakos commented 10 months ago

But I'm pretty sure it will not solve the problem for build the Addressables via Addressables Group window.

HaraldCsaszar commented 10 months ago

@sandolkakos Thanks for the info, always much appreciated. Unfortunately I'm afraid this will cover just the scenarios that are already covered by the SpineBuildPreprocessor being called during builds by subclassing BuildPlayerProcessor or IPreprocessBuildWithReport. If I misunderstood anything or my assumtions are wrong, please don't hesitate to let me know!

HaraldCsaszar commented 3 weeks ago

Issue has been reported on the forum where blend mode material's textures are not replaced alongside the main materials textures: https://esotericsoftware.com/forum/d/27224-full-size-textures-are-still-in-build-when-using-spineaddressables Can be reproduced with WhirlyBlendmodes example skeleton.

HaraldCsaszar commented 2 weeks ago

Note that the above bugfix requires both an update of:

  1. the spine-unity 4.2 runtime package
  2. the Spine On-Demand Loading Extensions package This is because a skeletonDataAsset reference has been added to the OnDemandTextureLoader base class (subclassed by GenericOnDemandTextureLoader and AddressablesTextureLoder) which is part of the spine-unity runtime package.

In order for textures at blend mode materials to be properly swapped out to low-resolution placeholder textures, the AddressablesTextureLoder's Skeleton Data Asset property needs to be assigned. This is automatically done at newly created AddressablesTextureLoders or when hitting Regenerate, but can be manually assigned and saved otherwise.