godotjs / GodotJS

GodotJS - Add TypeScript/JavaScript Support for Godot 4.x with v8/QuickJS
MIT License
195 stars 6 forks source link

multithreading support #22

Open GeTechG opened 4 days ago

GeTechG commented 4 days ago

I wanted to load resources in the background

ResourceLoader.load_threaded_request("res://main.tscn", "", false);

but my game crashed, but boon was the message)

ERROR: multi-threaded call not supported yet
   at: check_internal_state (modules/GodotJS/bridge/jsb_environment.h:276)

I'd like to ask how hard it is to add support, and if it can be done on the Godot side since thread execution is not required in GodotJs or am I missing something?

ialex32x commented 3 days ago

I'll try to fix this case right away. But overall, this part still needs to be finished/improved in future versions (finding a simple but robust solution, especially for isolated threaded execution).

ialex32x commented 3 days ago

I tested on my side with similar logic but did not encounter this check. Could you please provide more details about the loader and loaded scene? Or directly debug your project in play mode with launch arguments like --path "YOUR_PROJECT_FULL_PATH" --verbose res://main.tscn in your IDE/debugger.

My test case contains 2 scenes, and a script attached on each scene. main.tscn with MainControl.ts:

import { Error as GDError, Node2D, PackedScene, ResourceLoader } from "godot";

function load_scene_async(path: string) {
    return new Promise<PackedScene | null>(resolve => {
        let err = ResourceLoader.load_threaded_request(path, "", true);
        console.log(path, GDError[err]);

        let timerid: NodeJS.Timeout;
        timerid = setInterval(() => {
            let status = ResourceLoader.load_threaded_get_status(path);
            if (status != ResourceLoader.ThreadLoadStatus.THREAD_LOAD_IN_PROGRESS) {
                clearInterval(timerid);
                if (status == ResourceLoader.ThreadLoadStatus.THREAD_LOAD_LOADED) {
                    resolve(<PackedScene>ResourceLoader.load(path));
                } else {
                    resolve(null);
                }
            }
        }, 20);
    })
}

export default class MainControl extends Node2D {
    async _ready() {
        console.log(MainControl.name, "ready");
        let path = "res://secondary.tscn";

        let scene = await load_scene_async(path);
        if (scene) {
            let node = scene.instantiate();
            this.add_child(node);
        }
    }
}

secondary.tscn with SecondaryControl.ts:

import { Node2D } from "godot";

export default class SecondaryControl extends Node2D {
    _ready(): void {
        console.log(SecondaryControl.name, "ready");
    }
}
GeTechG commented 3 days ago

Hmm, I don't seem to see much difference, I also execute inside the promise, here is my code I specifically commented out that it falls on load.

export default class AssetsPromiseLoader extends Node {
    private promises: AssetsPromiseData[] = [];

    public load(items: LoadItem[], onProgress?: ProgressCallback): Promise<any[]> {
        return new Promise((resolve, reject) => {
            for (let item of items) {
                ResourceLoader.load_threaded_request(item.url, "", false);
            }
            this.promises.push({resolve, reject, items, onProgress});
        });
    }

    _process(_delta: float64) {
        // const deletePromises: number[] = [];
        // for (let i = 0; i < this.promises.length; i++) {
        //  const data = this.promises[i];
        //  let loadedCount = 0;
        //  let fullProgress = 0;
        //  for (let item of data.items) {
        //      const progress = new GArray<number>();
        //      const status = ResourceLoader.load_threaded_get_status(item.url, progress);
        //      fullProgress += progress.get_indexed(0);
        //      switch (status) {
        //          case ResourceLoader.ThreadLoadStatus.THREAD_LOAD_INVALID_RESOURCE:
        //              data.reject(`Invalid resource: ${item.url}`);
        //              deletePromises.push(i);
        //              break;
        //          case ResourceLoader.ThreadLoadStatus.THREAD_LOAD_FAILED:
        //              data.reject(`Failed to load: ${item.url}`);
        //              deletePromises.push(i);
        //              break;
        //          case ResourceLoader.ThreadLoadStatus.THREAD_LOAD_LOADED:
        //              loadedCount++;
        //              break;
        //      }
        //  }
        //  if (data.onProgress) {
        //      data.onProgress(fullProgress / data.items.length);
        //  }
        //  if (loadedCount === data.items.length) {
        //      data.resolve(data.items.map(item => ResourceLoader.load_threaded_get(item.url)));
        //      deletePromises.push(i);
        //  }
        // }
        // this.promises = this.promises.filter((_, i) => !deletePromises.includes(i));
    }
}
ialex32x commented 3 days ago

I also think there are no essential differences in the sample code. I'll try it again on other platforms. BTW, could you provide the stacktrace of the assertion failure by debugging your project in play mode (or from the log file of godot)? And are there any additional build options on your side for scons?

GeTechG commented 3 days ago

OS: linux (mint) If I understand how to run it correctly, that's all it gives me.

Log Initialize godot-rust (API v4.3.stable.official, runtime v4.3.stable.custom_build) Godot Engine v4.3.stable.custom_build.77dcf97d8 - https://godotengine.org TextServer: Added interface "Dummy" TextServer: Added interface "ICU / HarfBuzz / Graphite (Built-in)" JoypadLinux: udev enabled and loaded successfully. Xshape 1.1 detected. Xinerama 1.1 detected. Xrandr 1.6 detected. Xrender 0.11 detected. Xinput 2.2 detected. XInput: Refreshing devices. XInput: No touch devices found. WARNING: XOpenIM failed at: DisplayServerX11 (platform/linuxbsd/x11/display_server_x11.cpp:6103) WARNING: GENERAL - Message Id Number: 0 | Message Id Name: Loader Message terminator_CreateInstance: Failed to CreateInstance in ICD 3. Skipping ICD. Objects - 1 Object[0] - VK_OBJECT_TYPE_INSTANCE, Handle 93894741115232 at: _debug_messenger_callback (drivers/vulkan/rendering_context_driver_vulkan.cpp:305) WARNING: XCreateIC couldn't create wd.xic at: _create_window (platform/linuxbsd/x11/display_server_x11.cpp:5708) Devices: #0: NVIDIA NVIDIA GeForce RTX 4060 Ti - Supported, Discrete #1: Unknown llvmpipe (LLVM 15.0.7, 256 bits) - Supported, CPU - Vulkan Variable Rate Shading supported: Pipeline fragment shading rate Primitive fragment shading rate Attachment fragment shading rate, min texel size: (16, 16), max texel size: (16, 16), max fragment size: (4, 4) - Vulkan multiview supported: max view count: 32 max instances: 134217727 - Vulkan subgroup: size: 32 min size: 32 max size: 32 stages: STAGE_VERTEX, STAGE_TESSELLATION_CONTROL, STAGE_TESSELLATION_EVALUATION, STAGE_GEOMETRY, STAGE_FRAGMENT, STAGE_COMPUTE, STAGE_RAYGEN_KHR, STAGE_ANY_HIT_KHR, STAGE_CLOSEST_HIT_KHR, STAGE_MISS_KHR, STAGE_INTERSECTION_KHR, STAGE_CALLABLE_KHR, STAGE_TASK_NV, STAGE_MESH_NV supported ops: FEATURE_BASIC, FEATURE_VOTE, FEATURE_ARITHMETIC, FEATURE_BALLOT, FEATURE_SHUFFLE, FEATURE_SHUFFLE_RELATIVE, FEATURE_CLUSTERED, FEATURE_QUAD, FEATURE_PARTITIONED_NV quad operations in all stages Vulkan 1.3.277 - Forward Mobile - Using Device #0: NVIDIA - NVIDIA GeForce RTX 4060 Ti Startup PSO cache (0.8 MiB) XcursorGetTheme could not get cursor theme ScreenSaver: DBus 1.12.20 detected. FreeDesktopScreenSaver: Acquired screensaver inhibition cookie: 1494831593 PortalDesktop: DBus 1.12.20 detected. Using "default" pen tablet driver... Creating VMA small objects pool for memory type index 1 Shader 'CanvasSdfShaderRD' (group 0) SHA256: f65579ae334cdfb6901aee2527afd18ed626d1067a35c91f12fb632b45ce7f21 Shader 'SkeletonShaderRD' (group 0) SHA256: 4964fd59acf4406110ae7bca4b716f23ae52cc19864e21a8a35aee53e141e17e Shader 'SortShaderRD' (group 0) SHA256: 0b1e36114ab5330dc340cc740b0b946ed2dbf43098119b8d29cfa0222da18b7a Shader 'ParticlesShaderRD' (group 0) SHA256: 47fcf57848349fdf54197e5c1f64750a69c8d54787d3509ade4781c9faa31654 Shader 'ParticlesCopyShaderRD' (group 0) SHA256: 69566a7b0235d75ec40f504cd5555856aace22b5273899269166fde57287d26e Shader 'CanvasShaderRD' (group 0) SHA256: 165f6964cfc63fea91c4221f04e7b8a54b5dc39d76b5a57aed5f99b20050b148 Shader 'CanvasOcclusionShaderRD' (group 0) SHA256: 42f2245b35206bcace1cda9e39e9282519e9967978ba6d5956472809525b1150 Shader 'SceneForwardMobileShaderRD' (group 0) SHA256: e902df56af9ca9d9d9416d71378d35cb342cbc94502027ef4929feede81891a9 Shader 'SkyShaderRD' (group 0) SHA256: 4bc541bfde1c30032df77bb4c98974909ad0368d22557f44e6cf095b26c0490a Shader 'BokehDofRasterShaderRD' (group 0) SHA256: f30ffa9b63b7111cdeebc9cfd845e16bce37fb3def0098b0ea4291c5cba4c56c Shader 'BlurRasterShaderRD' (group 0) SHA256: 1491fde07959e0231c42129c86e0b36750d05d340fa12a234e747a34c291ab85 Shader 'CopyShaderRD' (group 0) SHA256: 4a43f34799f84bc5a387aff816d36330149d91299172af9be2d3ab05cc0c4e9d Shader 'CopyToFbShaderRD' (group 0) SHA256: 8668028c5ceed89276611359329de4a0f5d40702546caecc31456f63f6f9ffe8 Shader 'CubeToDpShaderRD' (group 0) SHA256: e3db5adc31b15e80112f4d7497dc8563c39b7d64675a53dcce7f9511c5ca9f80 Shader 'CubemapDownsamplerRasterShaderRD' (group 0) SHA256: 7b25ae2b6822705c2714ecbc0e14d2af08ab7d08e88a1c3b7fa0ecc9bb281250 Shader 'CubemapFilterRasterShaderRD' (group 0) SHA256: 41fc5058a6a86412a99e75cfc00cb64be45eb0aa6d46278a0a0355ed01bc5487 Shader 'CubemapRoughnessRasterShaderRD' (group 0) SHA256: 2b317415ef9a47a49583fda31b04795e8751db94b488132f8816daa29362958a Shader 'SpecularMergeShaderRD' (group 0) SHA256: a119881bb41d0ec2f36a7f060866ec38b88afa6d0e484a431c65b35b5a2e820b Shader 'ShadowFrustumShaderRD' (group 0) SHA256: c9c1b6421f8f6625effc9ddfa4bb2c09cafda0942b62fb8e1414e90820e18fa0 Shader 'MotionVectorsShaderRD' (group 0) SHA256: cbf3cbdcba0d0a2026c9fbf248556f1acd11beacc38a2e8c5ce7009ee49a1560 Shader 'LuminanceReduceRasterShaderRD' (group 0) SHA256: 3ccd96a23701d47b6b28640ac3624f391cf503bff4453adcf9486ce1f753c572 Shader 'TonemapShaderRD' (group 0) SHA256: 1c4748b3de350503048a1f9eca4f1dc2b308086460692ea0a81bababfd80542a Shader 'VrsShaderRD' (group 0) SHA256: def705023965d273a669c1ac6acfd7494138bde25b3e3bde0969c3c7a32c3aae Shader 'BlitShaderRD' (group 0) SHA256: 09b63e5347ccb6779fd79e54b093cfc915ec80510efa1f613c9ada9ff2e0f70d PulseAudio 15.99.0 detected. PulseAudio: context other PulseAudio: context other PulseAudio: context other PulseAudio: context ready PulseAudio: Detecting channels for device: alsa_output.pci-0000_01_00.1.hdmi-stereo-extra1 PulseAudio: detected 2 output channels PulseAudio: audio buffer frames: 512 calculated output latency: 11ms Using present mode: Enabled TextServer: Primary interface set to: "ICU / HarfBuzz / Graphite (Built-in)". [jsb][Verbose] v8 version: 12.4.254.21 [jsb][Verbose] EditorSettings is not available when initialising jsb::internal::Settings [JSDebugger][Debug] devtools://devtools/bundled/inspector.html?v8only=true&ws=127.0.0.1:9229/1 [jsb][Verbose] add search path: res://.godot/GodotJS [jsb][Verbose] add search path: res:// [jsb][Verbose] add search path: res://node_modules CORE API HASH: 3088492228 EDITOR API HASH: 444927587 [jsb][Verbose] checked file path res://.godot/GodotJS/src/__assets_promise_loader.js [jsb][Verbose] instantiating module res://.godot/GodotJS/src/__assets_promise_loader.js [jsb][Verbose] load main module res://.godot/GodotJS/src/__assets_promise_loader.js [jsb][Log] [EXPERIMENTAL] res://.godot/GodotJS/src/__assets_promise_loader.js script: 64962 inherits super: 0 native: 59138 [jsb][Verbose] create instance 93894793880960 of Node(64962) [jsb][Verbose] cross binding from C++. Node(59138) Loaded system CA certificates [jsb][Verbose] checked file path res://.godot/GodotJS/src/loader.js [jsb][Verbose] instantiating module res://.godot/GodotJS/src/loader.js [jsb][Verbose] checked file path res://.godot/GodotJS/src/__utils.js [jsb][Verbose] instantiating module res://.godot/GodotJS/src/__utils.js [jsb][Log] [EXPERIMENTAL] res://.godot/GodotJS/src/loader.js script: 64898 inherits super: 0 native: 59010 [jsb][Verbose] create instance 93894793909280 of Control(64898) [jsb][Verbose] cross binding from C++. Control(59010) Using present mode: Enabled ERROR: multi-threaded call not supported yet at: check_internal_state (modules/GodotJS/bridge/jsb_environment.h:277) ERROR: multi-threaded call not supported yet at: check_internal_state (modules/GodotJS/bridge/jsb_environment.h:277)

godot.log

I don't use anything extra I have a clean build of godot 4.3. I build the editor like this:

scons platform=linuxbsd target=editor
ialex32x commented 11 hours ago

I tried more platforms but still can't reproduce this problem (both on macos and linux mint 22, with godot branch 4.3 and tag 4.3-stable). However, it must be triggered by GodotJSScript::ensure_module_loaded() unexpectedly from somewhere. Can you debug from source via VSCode (or other IDE/debugger you have) to get the exact stacktrace of this assertion failure?

For VSCode, this godot docs may help. And for convenience, here is my config to debug in VSCode on Linux Mint:

tasks.json

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build",
            "group": "build",
            "type": "shell",
            "command": "scons",
            "args": [
              "p=linuxbsd",
              // enable this if v8 not available
              // "use_quickjs=yes",
              "compiledb=yes",
              "dev_build=yes",
              "target=editor"
            ],
            "problemMatcher": "$msCompile"
          }
    ]
}

launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb",
            "request": "launch",
            "name": "CodeLLDB launch",
            // path to your built godot executable file
            "program": "${workspaceFolder}/bin/godot.linuxbsd.editor.dev.x86_64",
            "args": [ 
                "res://main.tscn", 
                "--rendering-driver", "opengl3",
                "--path", 
                "!!!FULL_PATH_TO_YOUR_PROJECT!!!", 
                "--verbose" 
            ],
            "cwd": "!!!FULL_PATH_TO_YOUR_PROJECT!!!", 
            "externalConsole": false, 
            "preLaunchTask": "build"
        }
    ]
}

The exact Call Stack is available in Debug in VSCode on the breakpoint of the assertion failure triggered.

GeTechG commented 5 hours ago

I don't understand anything... I built as you have, I added a config for debug, I started godot to test and it worked, I checked on my old build also works, it somehow fixed itself, I'll try to look for something else, maybe I missed something.