[EgoVenture] Enhance scene cache by adding target scenes of hotspots

[ThmsKnz]

Please make sure you talk to the community before creating an issue.

Project:

• [X] Egoventure

Is your feature request related to a problem? Please describe. EgoVenture's cache is depending on the numbering of scenes. However it is not always possible to connect the scenes in this way. Take for example Carol Refurbished where man05.tscn includes a walk hotspot to man14.tscn.

Describe the solution you'd like I would propose to add the scenes to the cache that are listed in the hotspots of the scene passed to update_cache().

Additional context Here's a small prototype how such a feature could be added in scene_cache.gd. Such logic could get added in function update_cache() merging it with the removal/adding of scenes based on the index sequence.

func _add_linked_scenes(current_scene: String):
    var scene_node = get_scene(current_scene).instance()
    for node in _get_all_children(scene_node):
        if node.get_class() == "TextureButton" and "target_scene" in node:
            if node.target_scene != "":
                if not node.target_scene in _cache.keys() and \
                        not node.target_scene in _resource_queue.pending.keys():
                    _resource_queue.queue_resource(node.target_scene)
                    _queued_items.append(node.target_scene)

func _get_all_children(node:Node)->Array:
    var nodes: Array
    for child in node.get_children():
        nodes.append(child)
        if child.get_child_count() > 0:
            nodes.append_array(_get_all_children(child))
    return nodes

[MikaelNyqvist]

I've been using various ways to get around this (although none of them has been especially successful). I've never brought up any suggestion about including scenes that break chronology in the caching system, since I didn't think it was possible. If this works, it will be an excellent addition.

[ThmsKnz]

I've done some testing with the prototype and extended it to also scan the script of the current scene for target scenes (in a simple way without excluding comments).

I found it quite efficient to have both caching concepts in parallel:

• caching by the number sequence
• plus caching of target scenes of the current scene

In order to be even more efficient the concept could get extended that the result of the dynamic retrieval of the target scenes of the current scene is getting cached as well, but I haven't added this to the prototype yet.

Find below the three changes I've applied for testing the extended caching concept. Note: It is possible to test the pure new caching concept (without caching by the number sequence) by setting scene_count in configuration.tres to a negative value. Setting scene_count to 0 caches in addition the scenes having the same index number than the current scene,

1) Here's the prototype code that is to be added at the end of scene_cache.gd:

func _get_linked_scenes(current_scene: String) -> Array:
    var linked_scenes: Array
    var scene_node = get_scene(current_scene).instance()

    var regex = RegEx.new()
    regex.compile("res:\\/\\/[\\w\\/]*.tscn") # Regex for scene names (including scenes in comments)

    # get all target scenes listed in Hotspots
    for node in _get_all_children(scene_node):
        if node.get_class() == "TextureButton" and "target_scene" in node:
            var scene_path = node.target_scene
            if (
                scene_path != "" 
                and ResourceLoader.exists(scene_path)
                and not scene_path in linked_scenes
            ):
                linked_scenes.append(scene_path)

    # get all target scenes mentioned in Coding ("res://*.tscn")
    var scene_script = scene_node.get_script()
    if scene_script and scene_script.has_source_code():
        var scene_matches = regex.search_all(scene_script.source_code)
        for scene in scene_matches:
            var scene_path = scene.get_string()
            if ResourceLoader.exists(scene_path) and not scene_path in linked_scenes:
                linked_scenes.append(scene_path)

    return linked_scenes

func _get_all_children(node:Node)->Array:
    var nodes: Array
    for child in node.get_children():
        nodes.append(child)
        if child.get_child_count() > 0:
            nodes.append_array(_get_all_children(child))
    return nodes

2) The update_cache() function in scene_cache.gd needs to get changed in the following way: a) Insert the 7 lines below between return 1 and var scene_index = ...

        return 1

    var linked_scenes = _get_linked_scenes(current_scene)
    for linked_scene in linked_scenes:
        if not linked_scene in _cache.keys() and \
                not linked_scene in _resource_queue.pending.keys():
            print("Queueing load of linked scene %s" % linked_scene.get_file())
            _resource_queue.queue_resource(linked_scene)
            _queued_items.append(linked_scene)

    var scene_index = _get_index_from_filename(current_scene)

b) Change line if not cache_item in _permanent_cache: to if not cache_item in _permanent_cache and not cache_item in linked_scenes:

3) Currently EgoVenture is only updating the cache for FourSideRooms and EightSideRooms. So in order to fully test the extended caching concept it is required to add update_cache(path) at the end of the EgoVenture.change_scene() function. (It could then get removed from FourSideRoom and EightSideRoom to prevent that update_cache gets called twice.)

It's still a prototype and can definitely be further improved. Caching by 'scanning hotspots and scene scripts' has its limitations (e.g. only static target scenes are detected, coding in other scripts is not taken into account), but I think it's worth to take it into consideration. Let me know your thoughts whether you think this feature should get added.

[dploeger]

That's a very interesting thing and totally worth implementing. The only thing I'd like to add to your consideration is the fact, that mobile devices have straight memory limits. My old iPad shuts the game down if it takes more than 2GB (not sure about that number) of memory.

So, the optimal case would be that we could monitor memory usage and limit and act accordingly (I'm not sure if that's technically possible with Godot currently). Another option would be to optimize caching settings (turn off link cache and reduce cached scenes) for specific targets. This is possible when using the override feature of ProjectSettings.

[ThmsKnz]

Sorry, that it took so long to reply. You're right. It's important to watch the memory. One approach to optimize caching from a memory consumption point of view would be to 'only' cache target scenes that can be directly reached from a scene and not to cache scenes based on numbering.

Finding the linked scenes of a scene could be optimized by using the algorithm above (that reads hotspots and scans the code) only once for a scene and to keep the result in memory (during game runtime) or to even persist it.

This would also allow to extend caching (if needed) by adding all linked scenes of all linked scenes to the cache. But I'm not sure whether there's much performance gained from this. This rather leads to many unnecessary scenes that would get cached.

Another thought coming to my mind is to change the way how scenes get removed from the cache. Doing a full cleanup of all non-linked scenes after each scene change would be inefficient. It would be good to keep them a bit longer (for example only removing scenes that were not re-cached after some scene changes.

I don't know about Godot's intrinsic scene caching (at least I assume that Godot is doing something like this)... and I'm not sure whether there's a change in caching between Godot 3 and 4.

(Maybe it's best to have a chat whether it makes sense to continue these lines of thought :-)

[dploeger]

Yes, we should definitely enhance scene caching. It's too inflexible right now.

There's no real scene caching in Godot, because it's not really designed to be used like how we do it. For most games one scene is one level. Especially since we're using large images, caching is crucial to have a fluent gameplay.

I like your idea to walk through the scenes and cache the adjacent scenes. Maybe this could be an editor plugin. Something to run from time to time or to be triggered manually. It will scan all scenes and store the required resources in resource files, that can be loaded by the cache in the specific scene. The cache should be aware of the available memory (I don't know if this is maybe possible in Godot 4) and load/unload accordingly.

Do you think, you can work on that?

It's a rather big change.