Give more control and information to the developer when loading resources

[yaustar]

Related issues:

737 Asset loader should retry failed assets

494 assets.load and assets.loadFromUrl to return XHR object if one created internally

1246 Asset progress event support

Currently the resource loader is not resilient to unstable networks (such as mobile) and doesn't give much control or information (such as progress) back to the client.

This makes it very difficult for the client to know if a load has stalled or going very slowly to make the decision to retry or notify the user if the network connection has dropped.

Taking pc.SceneRegistry.loadSceneHierarchy for example: https://github.com/playcanvas/engine/blob/master/src/framework/scene-registry.js#L144

    SceneRegistry.prototype.loadSceneHierarchy = function (url, callback) {
        var self = this;

        // Because we need to load scripts before we instance the hierarchy (i.e. before we create script components)
        // Split loading into load and open
        var handler = this._app.loader.getHandler("hierarchy");

        // include asset prefix if present
        if (this._app.assets && this._app.assets.prefix && !pc.ABSOLUTE_URL.test(url)) {
            url = pc.path.join(this._app.assets.prefix, url);
        }

        handler.load(url, function (err, data) {
            if (err) {
                if (callback) callback(err);
                return;
            }

            // called after scripts are preloaded
            var _loaded = function () {
                self._app.systems.script.preloading = true;
                var entity = handler.open(url, data);
                self._app.systems.script.preloading = false;

                // clear from cache because this data is modified by entity operations (e.g. destroy)
                self._app.loader.clearCache(url, "hierarchy");

                // add to hierarchy
                self._app.root.addChild(entity);

                // initialize components
                pc.ComponentSystem.initialize(entity);
                pc.ComponentSystem.postInitialize(entity);

                if (callback) callback(err, entity);
            };

            // load priority and referenced scripts before opening scene
            self._app._preloadScripts(data, _loaded);
        });
    };

The request is made to the handler which makes the call to pc.HierarchyHandler.load https://github.com/playcanvas/engine/blob/master/src/resources/hierarchy.js#L10

        load: function (url, callback) {
            if (typeof url === 'string') {
                url = {
                    load: url,
                    original: url
                };
            }

            var assets = this._app.assets;

            pc.http.get(url.load, {
                retry: this.retryRequests
            }, function (err, response) {
                if (!err) {
                    pc.TemplateUtils.waitForTemplatesInScene(
                        response,
                        assets,
                        callback);
                } else {
                    var errMsg = 'Error while loading scene ' + url.original;
                    if (err.message) {
                        errMsg += ': ' + err.message;
                        if (err.stack) {
                            errMsg += '\n' + err.stack;
                        }
                    } else {
                        errMsg += ': ' + err;
                    }

                    callback(errMsg);
                }
            });
        },

But it (and other handlers) do not handle or return the XHR object which could be used to check progress and abort the request if need be.

Proposals:

• [ ] Any resource handler should return a handle that wraps the XHR object to allow the client to abort a load
• [ ] Any public API that does a load request should return this handle
• [ ] The err returned in the callback (example) should be an object with an error code and string rather than just a string when an error occurs.

Questions:

• Should the engine track progress as well or should that be optional for the client?
• Should the engine retry requests as #737 mentions or be optional for the client?
• Should the handle object be able to do the abort or should there be a matching cancel* function to any load* function?

[vkalpias]

Some resource loaders support retrying. Perhaps this needs to be implemented for scenes as well. See pc.ResourceLoader#enableRetry and where it's used in the engine.

[yaustar]

The issue is that on a bad connection, it doesn't actually error out so not sure if retry would work in this case. Part of the problem is that the developer can't query the progress of a download to communicate back to the player that's its being slow or that there are network issues.

[yaustar]

Here's an idea of what it could be like: https://github.com/playcanvas/engine/pull/2025

Starting to wonder if the scene registry should be a special case and that for assets, it would be part of the asset object/class and handled by the asset registry.

[yaustar]

Thinking about this more, maybe it's enough give users an API to cancel a load (e.g this.app.assets.cancelLoad)?

Edit: @vkalpias Would there be any problems on setting the default for retries to be true? This may help make the preloader be a bit more robust if an asset fails/stalls loading.

[marklundin]

Would love a bit of a rethink of the loading api. Currently it still seems quite a little convoluted and hidden away. It would be good for example to leverage some of the more modern loading strategies and allow the developer more control over the process, think server workers and cache api

[Maksims]

Service workers - are not related to application logic, it is up to a developer to implement their service workers with handling fetch requests. Caching - this is server side logic, and currently engine by default does a good job of adding a hash to assets, allowing to use of hard-cache policies by server-side and revalidate if the asset file has been changed (hash will change).

[marklundin]

I'm not really thinking about the browser cache per se, more so caching strategies that are entirely application logic. I have use cases where I have multiple scenes which are being loaded and unloaded and it's not clear what happens with them. Do they remain in memory, browser cache, or in CacheStorage. Granted, I can look under the hood and find this out, but it would be much better for a developer to have control and overview of this if for example I had assets shared across scenes.

Of course, you can forego the loading api and add assets manually, but in doing so, you need to pull lots of code out of the existing loading api, such as initialize(), postInitialize(), decompress the config etc. This may partially be a documentation issue, but it feels like there's room to improve the developer experience here.

An example use case I've had is to preload 2 scenes, and play the first one as soon as it has loaded. However if the second scene has not loaded, you are blocked from starting scene 1 until the second scene returns because loadScene() initiates a new http request, and the browser caps connections. There are of course ways of getting around this, but it's not obvious to the developer what's going on. Also, I understand there is a newer loadSceneData which solves this issue, but the semantics could be clearer around what these loading apis' do. For example loadSceneData loads but doesn't run. loadSceneHeirachy() loads and then runs but doesn't apply scene settings, loadScene() does something else, but should not be used? Ideally, there should be better separation between loading a scene and applying a scene. A developer should be able to have control over where or not the scene settings are applied, and it should be clear how and when this will occur.

Appreciate this I'm slightly broadening the original proposal

[yaustar]

For example loadSceneData loads but doesn't run. loadSceneHeirachy() loads and then runs but doesn't apply scene settings, loadScene() does something else, but should not be used? Ideally, there should be better separation between loading a scene and applying a scene. A developer should be able to have control over where or not the scene settings are applied, and it should be clear how and when this will occur.

I've got a ticket in the developer documentation repo to explain this fully. TLDR:

loadSceneData does a network request for the scene JSON data and caches the data in the engine until unloadSceneData is called loadSceneHeirachy will do a network request for scene JSON data unless it is already cached. It doesn't cache any data meaning if it wasn't cached before, the next time it is called for the same scene, it will do another network request. Whether it is cached in the browser is up to the browser. It will then create the entities of the loaded scene and adds them to the current app hierarchy and calls initialises all the entities. loadSceneSettings does the same as loadSceneHeirachy except applies scene settings.

The naming isn't great but currently confined with not being able to change existing public API.

loadScene should not be used unless you are using engine only and will to call private API to initialise components after the scene has been loaded and entities created and quite frankly a bad name that we can't change and wasn't keen on it becoming public.

Ideally I would like to have loadSceneAll that loads both the scene heirachy and settings (same logic as loadSceneHeirachy and loadSceneSettings) but fear of confusing the API.

but it would be much better for a developer to have control and overview of this if for example I had assets shared across scenes.

We should have 'best' practise tutorial/user manual page for this or at least one approach. I will add it to the task list.

[yaustar]

An example use case I've had is to preload 2 scenes, and play the first one as soon as it has loaded. However if the second scene has not loaded, you are blocked from starting scene 1 until the second scene returns because loadScene() initiates a new http request, and the browser caps connections.

I'm a bit surprised that this is the behaviour as Scene 1 should be able to start (create entities) after it has finished loading regardless of what else is being loaded from network. loadSceneHierarchy and Settings don't depend on a network load if the data is cached by loadSceneData.

[Maksims]

Scenes API could be improved, indeed.

Regarding assets - they do not unload automatically, and currently, it is up to a developer and application logic to decide when and what to unload. There is a lot of different scenarios and use cases. Coming up with a generic strategy for assets unloading for me seems not reachable. Definitely open to hearing if there is a solution that can cover everyone.

I believe a good list of tools and utilities can help the developer to develop own unloading strategy, for example

1. Asset reference counters.
2. Asset tags (already exist).

Tags - are a great tool to mark assets, and queries can nicely filter them for loading/unloading. Reference counter - can be a complex task, but definitely would help a lot as well. Unfortunately it is not implemented in engine atm.

[marklundin]

Yes we’ve used tags before to reference assets required by a given scene, however it’s a little fragile as it requires an artist to intentionally tag every asset which is very easy to miss. Also, when you load a scene using tagged assets and it fails because someone has missed one, it’s not obvious that is what’s happened and you need to manually traverse the entire scene hierarchy to find the missing tag.

I actually like the ability to additively load a scene, but it would be nice to have a way to automatically tag any asset that belongs to a given scene. It would reduce artist/developer error as it’s similar to the default enabled ‘preload’ flag. Also it allows the developer to explicitly state what to load and when ‘load([ ...sceneA, ...sceneB ])’

[marklundin]

One thing I’ve also noticed is that the config files often changes as the editor updates and this is not always backward compatible with a given engine version. It would be good to have a versioning to the config and to check for this when loading one

[yaustar]

Regarding assets used by a scene, I actually would prefer this to be on the editor side where it should be possible to show/filter all assets used in the current scene. It won't cover assets loaded by code via the registry but would allow a user to add a tag to all the assets at once.

[marklundin]

Following up, I think it would be good at a minimum to expose the underlying XHR object to users. This would allow users to implement things like percentage based loading progress etc. Any thoughts with us doing this?