search

Adriankhl / godot-llm

LLM in Godot
MIT License
100 stars 3 forks source link

Editor crashes when using arch linux #12

Closed KMontag42 closed 1 month ago

KMontag42 commented 2 months ago 

❯ godot -e
Godot Engine v4.2.2.stable.arch_linux - https://godotengine.org
OpenGL API 3.3.0 NVIDIA 550.78 - Compatibility - Using Device: NVIDIA - NVIDIA GeForce RTX 3090

WARNING: Blend file import is enabled in the project settings, but no Blender path is configured in the editor settings. Blend files will not be imported.
     at: _editor_init (modules/gltf/register_types.cpp:63)
free(): invalid pointer
zsh: IOT instruction (core dumped)  godot -e```

any ideas here? it looks like there is something goofy going on, but I'm not entirely sure what it is.
Adriankhl commented 2 months ago

Core dump is always hard, but let's try.

The first strange I see is the OpenGL API 3.3.0, do you know why it is opengl3? Are you opening a new project or what? From all my settings (non-nvidia) they are either vulkan or opengl 4.

The development version has a bit more debug message there, can you replace the libgodot_llm.linux.release.x86_64.so by the file in godot_llm_linux.zip, then turn on Verbose stdout in your project setting or add the following to project.godot:

[debug]

settings/stdout/verbose_stdout=true

and then open the project and paste the log here?

Finally, the distributed so file is compiled on Ubuntu 22.04 vm, so that could be some incompatibility with the Arch libraries, though I have also tested it on my OpenSUSE which seems to work fine. Can you clone the repo and compile it from source, it has almost no dependency so it should be easy, something like this, assume you have cmake, ninja and the typicall gcc develop tools installed:

git clone https://github.com/Adriankhl/godot-llm.git
cd godot-llm
mkdir build
cd build
cmake .. -GNinja -DLLAMA_NATIVE=OFF -DCMAKE_BUILD_TYPE=Release
ninja -j6

Then you will see the compiled libgodot_llm.linux.release.x86_64.so file there, put it to your project and see if it works or not

KMontag42 commented 2 months ago

In the short term I switched to my windows install, but I will definitely spend some time debugging this over the next week or so. Thanks so much for the speedy reply!!

potassium-shot commented 2 months ago

Hello! I'm on manjaro and also have the issue, here are the logs with the dev build:

Godot Engine v4.2.2.stable.arch_linux - https://godotengine.org
TextServer: Added interface "Dummy"
TextServer: Added interface "ICU / HarfBuzz / Graphite (Built-in)"
Xshape 1.1 detected.
Xinerama 1.1 detected.
Xrandr 1.6 detected.
Xrender 0.11 detected.
Xinput 2.2 detected.
XInput: Refreshing devices.
XInput: No touch devices found.
Vulkan devices:
  #0: NVIDIA NVIDIA GeForce RTX 3070 Laptop GPU - Supported, Discrete
  #1: Intel Intel(R) UHD Graphics (CML GT2) - Supported, Integrated
Vulkan API 1.3.277 - Forward Mobile - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3070 Laptop GPU
Optional extension VK_KHR_incremental_present not found
Optional extension VK_GOOGLE_display_timing not found
- Vulkan Variable Rate Shading supported:
  Pipeline fragment shading rate
  Primitive fragment shading rate
  Attachment fragment shading rate, min texel size: (16, 16), max texel size: (16, 16)
- Vulkan multiview supported:
  max view count: 32
  max instances: 134217727
- Vulkan subgroup:
  size: 32
  min size: 32
  max size: 32
  stages: STAGE_VERTEX, STAGE_TESSELLATION_CONTROL, STAGE_TESSELLATION_EVALUATION, STAGE_GEOMETRY, STAGE_FRAGMENT, STAGE_COMPUTE, STAGE_RAYGEN_KHR, STAGE_ANY_HIT_KHR, STAGE_CLOSEST_HIT_KHR, STAGE_MISS_KHR, STAGE_INTERSECTION_KHR, STAGE_CALLABLE_KHR, STAGE_TASK_NV, STAGE_MESH_NV
  supported ops: FEATURE_BASIC, FEATURE_VOTE, FEATURE_ARITHMETIC, FEATURE_BALLOT, FEATURE_SHUFFLE, FEATURE_SHUFFLE_RELATIVE, FEATURE_CLUSTERED, FEATURE_QUAD, FEATURE_PARTITIONED_NV
  quad operations in all stages
Startup PSO cache (1.0 MiB)
ScreenSaver: DBus 1.14.10 detected.
PortalDesktop: DBus 1.14.10 detected.
Using "default" pen tablet driver...
Creating VMA small objects pool for memory type index 1
Shader 'CanvasSdfShaderRD' (group 0) SHA256: 192fa22dac1a8a7551aa1705da8750d884ae1b9c1019bdc26c2f354cd9085253
Shader 'SkeletonShaderRD' (group 0) SHA256: 92722a4f381fd3f2170e021dee744246f8afb22eae5b6436dc3bd76446c3a164
Shader 'SortShaderRD' (group 0) SHA256: 4b6e206709c0fbc6fe462a1455e1e27c4428702d3849fa9b182eb7da1b9c0f7f
Shader 'ParticlesShaderRD' (group 0) SHA256: 5d22dabcee81b3a4bc036c6eb4ea5058697b556f629a62d1f1683ccad105e242
Shader 'ParticlesCopyShaderRD' (group 0) SHA256: c04c87ccd6f4ebd46c32306aec3de1f9e427a322d83fc82bdcb96861b999bbb7
Shader 'CanvasShaderRD' (group 0) SHA256: 40e07b1ee0c2f8a45a77351b2a605b0cdb0f527fa1f6270a82514c69585eec20
Shader 'CanvasOcclusionShaderRD' (group 0) SHA256: 731cf7571ba2dbafedadf4498c730b4635f116aa8bb4c75674706cbda52dfe4d
Shader 'SceneForwardMobileShaderRD' (group 0) SHA256: 2d34d55eddee82393d3cac0a3113e95416f5c238d44a5c9b0ee5e35c3a9215aa
Shader 'SkyShaderRD' (group 0) SHA256: 88e46645d7003ffd2f1d93ec85b224badbfae226c5ccafaaad76d37d418bc26a
Shader 'BokehDofRasterShaderRD' (group 0) SHA256: ac6720735e8c4ff7bf543b437339bfbddad963b89ab7413ab4c9077067f4e688
Shader 'BlurRasterShaderRD' (group 0) SHA256: 4e73b7207c37eb983f2d677de2d060ed16de877a6553cbaf8a70081c6dd2af20
Shader 'CopyShaderRD' (group 0) SHA256: f628bf35b0a0c97fe07a42b0a5dd72fd34d5584481290141debb14879e946a7c
Shader 'CopyToFbShaderRD' (group 0) SHA256: a46a758e8cc8f0c76ab3e215fa40409bdd466abd6e003b304f42a700f39d5273
Shader 'CubeToDpShaderRD' (group 0) SHA256: 2979d125b1841385718a49352b792af29cea23164c1d723e4b056c917a3e8372
Shader 'CubemapDownsamplerRasterShaderRD' (group 0) SHA256: b78eb0f7875bef123614ea494380826020527c2c2e347ed62f5e9f0a5f99525e
Shader 'CubemapFilterRasterShaderRD' (group 0) SHA256: f90d4e362808274c4813b769f379b25fe302218c31dc20453c35033255056043
Shader 'CubemapRoughnessRasterShaderRD' (group 0) SHA256: e72b125650e1b2abb33417facf8881f06b622e46faae26d57b465fa58d2548ee
Shader 'SpecularMergeShaderRD' (group 0) SHA256: e089c7aab673b54f2532103ac70c45e861f84924c54545f7e51baba2ea1f679a
Shader 'ShadowFrustumShaderRD' (group 0) SHA256: ccf723ac05d73ab9eaa1ba85ac92b7ff32a17a8a37f4607d4aee659985693906
Shader 'MotionVectorsShaderRD' (group 0) SHA256: ebd35b07779c81a39dd9ddcd0454781f12533c5796a0666ffa010281ec0fbc3e
Shader 'LuminanceReduceRasterShaderRD' (group 0) SHA256: 04d1ef236e6c673eb2ac193e77ff35f4e345919ae1ee787a910e0492e358f2d8
Shader 'TonemapShaderRD' (group 0) SHA256: 4f5ab260c61e1c69627bddd36a33509db10d2a7fea9541bf288a438506b8178e
Shader 'VrsShaderRD' (group 0) SHA256: f1efc533285aeb0f4fdf307de923a4c2f3827c0a9d1d861577d5bcff6e8904cb
Shader 'BlitShaderRD' (group 0) SHA256: 79311bde404a354024c88187fd36926db28033a0f51d73cad5e8fc37dab5e6ac
JoypadLinux: udev enabled and loaded successfully.
PulseAudio 17.0.0 detected.
PulseAudio: context other
PulseAudio: context other
PulseAudio: context other
PulseAudio: context ready
PulseAudio: Detecting channels for device: alsa_output.pci-0000_00_1f.3.analog-stereo
PulseAudio: detected 2 output channels
PulseAudio: audio buffer frames: 512 calculated output latency: 11ms

Vulkan queue submit: Swapchain is out of date, recreating.
TextServer: Primary interface set to: "ICU / HarfBuzz / Graphite (Built-in)".
CORE API HASH: 2995524600
EDITOR API HASH: 3767149329
EditorSettings: Load OK!
Creating VMA small objects pool for memory type index 4
EditorSettings: Save OK!
free(): invalid pointer
zsh: IOT instruction (core dumped)  godot -e project.godot

I also tried building from source and using the built .so, but the exact same issue occurs (exact same logs)

Adriankhl commented 2 months ago

Hello! I'm on manjaro and also have the issue, here are the logs with the dev build:

Godot Engine v4.2.2.stable.arch_linux - https://godotengine.org
TextServer: Added interface "Dummy"
TextServer: Added interface "ICU / HarfBuzz / Graphite (Built-in)"
Xshape 1.1 detected.
Xinerama 1.1 detected.
Xrandr 1.6 detected.
Xrender 0.11 detected.
Xinput 2.2 detected.
XInput: Refreshing devices.
XInput: No touch devices found.
Vulkan devices:
  #0: NVIDIA NVIDIA GeForce RTX 3070 Laptop GPU - Supported, Discrete
  #1: Intel Intel(R) UHD Graphics (CML GT2) - Supported, Integrated
Vulkan API 1.3.277 - Forward Mobile - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3070 Laptop GPU
Optional extension VK_KHR_incremental_present not found
Optional extension VK_GOOGLE_display_timing not found
- Vulkan Variable Rate Shading supported:
  Pipeline fragment shading rate
  Primitive fragment shading rate
  Attachment fragment shading rate, min texel size: (16, 16), max texel size: (16, 16)
- Vulkan multiview supported:
  max view count: 32
  max instances: 134217727
- Vulkan subgroup:
  size: 32
  min size: 32
  max size: 32
  stages: STAGE_VERTEX, STAGE_TESSELLATION_CONTROL, STAGE_TESSELLATION_EVALUATION, STAGE_GEOMETRY, STAGE_FRAGMENT, STAGE_COMPUTE, STAGE_RAYGEN_KHR, STAGE_ANY_HIT_KHR, STAGE_CLOSEST_HIT_KHR, STAGE_MISS_KHR, STAGE_INTERSECTION_KHR, STAGE_CALLABLE_KHR, STAGE_TASK_NV, STAGE_MESH_NV
  supported ops: FEATURE_BASIC, FEATURE_VOTE, FEATURE_ARITHMETIC, FEATURE_BALLOT, FEATURE_SHUFFLE, FEATURE_SHUFFLE_RELATIVE, FEATURE_CLUSTERED, FEATURE_QUAD, FEATURE_PARTITIONED_NV
  quad operations in all stages
Startup PSO cache (1.0 MiB)
ScreenSaver: DBus 1.14.10 detected.
PortalDesktop: DBus 1.14.10 detected.
Using "default" pen tablet driver...
Creating VMA small objects pool for memory type index 1
Shader 'CanvasSdfShaderRD' (group 0) SHA256: 192fa22dac1a8a7551aa1705da8750d884ae1b9c1019bdc26c2f354cd9085253
Shader 'SkeletonShaderRD' (group 0) SHA256: 92722a4f381fd3f2170e021dee744246f8afb22eae5b6436dc3bd76446c3a164
Shader 'SortShaderRD' (group 0) SHA256: 4b6e206709c0fbc6fe462a1455e1e27c4428702d3849fa9b182eb7da1b9c0f7f
Shader 'ParticlesShaderRD' (group 0) SHA256: 5d22dabcee81b3a4bc036c6eb4ea5058697b556f629a62d1f1683ccad105e242
Shader 'ParticlesCopyShaderRD' (group 0) SHA256: c04c87ccd6f4ebd46c32306aec3de1f9e427a322d83fc82bdcb96861b999bbb7
Shader 'CanvasShaderRD' (group 0) SHA256: 40e07b1ee0c2f8a45a77351b2a605b0cdb0f527fa1f6270a82514c69585eec20
Shader 'CanvasOcclusionShaderRD' (group 0) SHA256: 731cf7571ba2dbafedadf4498c730b4635f116aa8bb4c75674706cbda52dfe4d
Shader 'SceneForwardMobileShaderRD' (group 0) SHA256: 2d34d55eddee82393d3cac0a3113e95416f5c238d44a5c9b0ee5e35c3a9215aa
Shader 'SkyShaderRD' (group 0) SHA256: 88e46645d7003ffd2f1d93ec85b224badbfae226c5ccafaaad76d37d418bc26a
Shader 'BokehDofRasterShaderRD' (group 0) SHA256: ac6720735e8c4ff7bf543b437339bfbddad963b89ab7413ab4c9077067f4e688
Shader 'BlurRasterShaderRD' (group 0) SHA256: 4e73b7207c37eb983f2d677de2d060ed16de877a6553cbaf8a70081c6dd2af20
Shader 'CopyShaderRD' (group 0) SHA256: f628bf35b0a0c97fe07a42b0a5dd72fd34d5584481290141debb14879e946a7c
Shader 'CopyToFbShaderRD' (group 0) SHA256: a46a758e8cc8f0c76ab3e215fa40409bdd466abd6e003b304f42a700f39d5273
Shader 'CubeToDpShaderRD' (group 0) SHA256: 2979d125b1841385718a49352b792af29cea23164c1d723e4b056c917a3e8372
Shader 'CubemapDownsamplerRasterShaderRD' (group 0) SHA256: b78eb0f7875bef123614ea494380826020527c2c2e347ed62f5e9f0a5f99525e
Shader 'CubemapFilterRasterShaderRD' (group 0) SHA256: f90d4e362808274c4813b769f379b25fe302218c31dc20453c35033255056043
Shader 'CubemapRoughnessRasterShaderRD' (group 0) SHA256: e72b125650e1b2abb33417facf8881f06b622e46faae26d57b465fa58d2548ee
Shader 'SpecularMergeShaderRD' (group 0) SHA256: e089c7aab673b54f2532103ac70c45e861f84924c54545f7e51baba2ea1f679a
Shader 'ShadowFrustumShaderRD' (group 0) SHA256: ccf723ac05d73ab9eaa1ba85ac92b7ff32a17a8a37f4607d4aee659985693906
Shader 'MotionVectorsShaderRD' (group 0) SHA256: ebd35b07779c81a39dd9ddcd0454781f12533c5796a0666ffa010281ec0fbc3e
Shader 'LuminanceReduceRasterShaderRD' (group 0) SHA256: 04d1ef236e6c673eb2ac193e77ff35f4e345919ae1ee787a910e0492e358f2d8
Shader 'TonemapShaderRD' (group 0) SHA256: 4f5ab260c61e1c69627bddd36a33509db10d2a7fea9541bf288a438506b8178e
Shader 'VrsShaderRD' (group 0) SHA256: f1efc533285aeb0f4fdf307de923a4c2f3827c0a9d1d861577d5bcff6e8904cb
Shader 'BlitShaderRD' (group 0) SHA256: 79311bde404a354024c88187fd36926db28033a0f51d73cad5e8fc37dab5e6ac
JoypadLinux: udev enabled and loaded successfully.
PulseAudio 17.0.0 detected.
PulseAudio: context other
PulseAudio: context other
PulseAudio: context other
PulseAudio: context ready
PulseAudio: Detecting channels for device: alsa_output.pci-0000_00_1f.3.analog-stereo
PulseAudio: detected 2 output channels
PulseAudio: audio buffer frames: 512 calculated output latency: 11ms

Vulkan queue submit: Swapchain is out of date, recreating.
TextServer: Primary interface set to: "ICU / HarfBuzz / Graphite (Built-in)".
CORE API HASH: 2995524600
EDITOR API HASH: 3767149329
EditorSettings: Load OK!
Creating VMA small objects pool for memory type index 4
EditorSettings: Save OK!
free(): invalid pointer
zsh: IOT instruction (core dumped)  godot -e project.godot

I also tried building from source and using the built .so, but the exact same issue occurs (exact same logs)

Hi, can you clarify whether you are running the cpu build or the vulkan build? And are you running the mono-version of Godot?

potassium-shot commented 2 months ago

When I originally had the issue, I was running the AssetLib version, not sure exactly if that's the CPU or the Vulkan version.

When running the dev version, I used the one you provided in this message, and for the manual build, I used the command you provided in the same message, which I'm guessing is CPU? And i am not running the mono version of Godot.

Adriankhl commented 2 months ago

@potassium-shot Can you try this build godot_linux_release.zip? Here I instantiate thread and mutex in a different place

potassium-shot commented 2 months ago

Hi @Adriankhl, sadly the build you gave me produces the same issue, but weirdly enough, I tried to build in debug this time, and now the editor launches, with godot-llm nodes available in the editor. Though, running the following code makes the project crash:

func _ready():
    var gdllama = GDLlama.new()
    gdllama.model_path = "res://models/Meta-Llama-3-8B-Instruct-IQ3_M.gguf"
    gdllama.n_predict = 20
    var generated_text = gdllama.generate_text_simple("Hello") # crash happens on this line
    print(generated_text)

with the following logs in the console:

Instantiate GDLlama mutex
Instantiate GDLlama thread
Instantiate GDLlama thread -- done
generate_text_simple
generate_text_mutex locked
generate_text_simple_internal
generate_text_common start
llama_model_loader: loaded meta data with 21 key-value pairs and 291 tensors from models/Meta-Llama-3-8B-Instruct-IQ3_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = Meta-Llama-3-8B-Instruct-imatrix
llama_model_loader: - kv   2:                          llama.block_count u32              = 32
llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
llama_model_loader: - kv   4:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   7:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   8:                       llama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                          general.file_type u32              = 27
llama_model_loader: - kv  11:                           llama.vocab_size u32              = 128256
llama_model_loader: - kv  12:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  16:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 128001
llama_model_loader: - kv  19:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
llama_model_loader: - kv  20:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q4_K:   68 tensors
llama_model_loader: - type q6_K:    1 tensors
llama_model_loader: - type iq3_s:  157 tensors
llm_load_vocab: missing pre-tokenizer type, using: 'default'
llm_load_vocab:
llm_load_vocab: ************************************
llm_load_vocab: GENERATION QUALITY WILL BE DEGRADED!
llm_load_vocab: CONSIDER REGENERATING THE MODEL
llm_load_vocab: ************************************
llm_load_vocab:
EditorSettings: Save OK!
GDLlama destructor
GDLlama destructor -- done
XR: Clearing primary interface
XR: Removed interface "Native mobile"
XR: Removed interface "OpenXR"
double free or corruption (!prev)
zsh: IOT instruction (core dumped)  godot -e project.godot

It's really weird, after trying a few times, the release build never runs, but the debug build always runs.

Adriankhl commented 2 months ago

@potassium-shot @KMontag42 I decided to create a Manjora live usb to reproduce the problem. And surprisingly, based on my investigation, the problem actually comes from the Arch build of Godot. The Godot program from the official website runs perfectly fine.

And yes, the plugin I attached earlier is problematic, you may try the to run the plugin in the asset library, or this one godot_linux_release.zip, with Godot from the official website.

potassium-shot commented 1 month ago

Indeed, after installing the official godot build, it works flawlessly! Thanks a lot for the time and effort, both in this plugin and in figuring out the issue, very much appreciated!

KMontag42 commented 1 month ago

Thank you very much for the investigation. I will close this thread as it works for me now!