search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
87.04k stars 19.54k forks source link

Large Scene with lots of meshes crashes when baking LightmapGI #86416

Closed jcostello closed 7 months ago

jcostello commented 7 months ago

Tested versions

Godot Engine v4.3.dev.custom_build [9d1cbab1c]

System information

Ubuntu with Nvidia 2060

Issue description

Baking a large scene with lot of meshes crashes when baking LightmapGI

[1] /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fc53f642520] (??:0)
[2] /lib/x86_64-linux-gnu/libc.so.6(+0x1a0e4a) [0x7fc53f7a0e4a] (??:0)                                                                                                [3] Image::initialize_data(int, int, bool, Image::Format) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/io/image.cpp:2187 (discriminator 4))
[4] Image::create_empty(int, int, bool, Image::Format) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/io/image.cpp:2153)             
[5] LightmapperRD::_blit_meshes_into_atlas(int, Vector<Ref<Image> >&, Vector<Ref<Image> >&, AABB&, Vector2i&, int&, bool (*)(float, String const&, void*, bool), void*
) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/modules/lightmapper_rd/lightmapper_rd.cpp:322 (discriminator 3))                        
[6] LightmapperRD::bake(Lightmapper::BakeQuality, bool, float, int, float, float, int, bool, bool, Lightmapper::GenerateProbes, Ref<Image> const&, Basis const&, bool 
(*)(float, String const&, void*, bool), void*, float) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/modules/lightmapper_rd/lightmapper_rd.cpp:1008)     [7] LightmapGI::bake(Node*, String, bool (*)(float, String const&, void*, bool), void*) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/3d/lightmap_
gi.cpp:1064)                                                                       
[8] LightmapGIEditorPlugin::_bake_select_file(String const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/editor/plugins/lightmap_gi_editor_plugin.cpp:
71)                                                                                                                                                                   
[9] LightmapGIEditorPlugin::_bake() (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/editor/plugins/lightmap_gi_editor_plugin.cpp:117)
[10] void call_with_variant_args_helper<__UnexistingClass>(__UnexistingClass*, void (__UnexistingClass::*)(), Variant const**, Callable::CallError&, IndexSequence<>) 
(/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/variant/binder_common.h:308 (discriminator 4))
[11] void call_with_variant_args_dv<__UnexistingClass>(__UnexistingClass*, void (__UnexistingClass::*)(), Variant const**, int, Callable::CallError&, Vector<Variant> 
const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/variant/binder_common.h:451)
[12] MethodBindT<>::call(Object*, Variant const**, int, Callable::CallError&) const (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/object/method_b
ind.h:335)
[13] Object::callp(StringName const&, Variant const**, int, Callable::CallError&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/object/object.cpp:7
75)
[14] Callable::callp(Variant const**, int, Variant&, Callable::CallError&) const (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/variant/callable.cpp
:69)
[15] Object::emit_signalp(StringName const&, Variant const**, int) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/object/object.cpp:1128)
[16] Node::emit_signalp(StringName const&, Variant const**, int) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/main/node.cpp:3642)
[17] Error Object::emit_signal<>(StringName const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/object/object.h:922)
[18] BaseButton::_pressed() (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/gui/base_button.cpp:139)
[19] BaseButton::on_action_event(Ref<InputEvent>) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/gui/base_button.cpp:177)
[20] BaseButton::gui_input(Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/gui/base_button.cpp:69)
[21] Control::_call_gui_input(Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/gui/control.cpp:1816)
[22] Viewport::_gui_call_input(Control*, Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/main/viewport.cpp:1596)
[23] Viewport::_gui_input_event(Ref<InputEvent>) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/main/viewport.cpp:1862 (discriminator 3))
[24] Viewport::push_input(Ref<InputEvent> const&, bool) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/main/viewport.cpp:3351)
[25] Window::_window_input(Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/scene/main/window.cpp:1580)
[26] void call_with_variant_args_helper<Window, Ref<InputEvent> const&, 0ul>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, Callable::CallError&,
 IndexSequence<0ul>) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/variant/binder_common.h:303 (discriminator 4))
[27] void call_with_variant_args<Window, Ref<InputEvent> const&>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, int, Callable::CallError&) (/medi
a/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/variant/binder_common.h:418)
[28] CallableCustomMethodPointer<Window, Ref<InputEvent> const&>::call(Variant const**, int, Variant&, Callable::CallError&) const (/media/juan/af3f89c8-199f-4946-828
1-6c6a4187273a/dev/godot/./core/object/callable_method_pointer.h:99)
[29] Callable::callp(Variant const**, int, Variant&, Callable::CallError&) const (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/variant/callable.cpp
:57)
[30] Variant Callable::call<Ref<InputEvent> >(Ref<InputEvent>) const (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/./core/variant/variant.h:851)
[31] DisplayServerX11::_dispatch_input_event(Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/platform/linuxbsd/x11/display_server_
x11.cpp:4001)
[32] DisplayServerX11::_dispatch_input_events(Ref<InputEvent> const&) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/platform/linuxbsd/x11/display_server
_x11.cpp:3978)
[33] Input::_parse_input_event_impl(Ref<InputEvent> const&, bool) (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/input/input.cpp:759)
[34] Input::flush_buffered_events() (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/core/input/input.cpp:1021)
[35] DisplayServerX11::process_events() (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/platform/linuxbsd/x11/display_server_x11.cpp:5072)
[36] OS_LinuxBSD::run() (/media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/platform/linuxbsd/os_linuxbsd.cpp:931)
[37] /media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/bin/godot.linuxbsd.editor.dev.x86_64(main+0x19f) [0x55f1e09ae2a8] (/media/juan/af3f89c8-199f-4946-8281
-6c6a4187273a/dev/godot/platform/linuxbsd/godot_linuxbsd.cpp:76)
[38] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fc53f629d90] (??:0)
[39] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7fc53f629e40] (??:0)
[40] /media/juan/af3f89c8-199f-4946-8281-6c6a4187273a/dev/godot/bin/godot.linuxbsd.editor.dev.x86_64(_start+0x25) [0x55f1e09ae045] (??:?)
-- END OF BACKTRACE -

Steps to reproduce

Use MRP

Bake lightmap as it is

Minimal reproduction project (MRP)

https://www.dropbox.com/scl/fi/owgcqzu68346pfvuubts3/MRP.zip?rlkey=oxyacpf62hb2f3n0xbup6wlcj&dl=0

bitsawer commented 7 months ago

Confirmed on current master at 9d1cbab1c432b6f1d66ec939445bec68b6af519e, looks like yet another integer overflow issue, this time in:

https://github.com/godotengine/godot/blob/13a0d6e9b253654f5cc2a44f3d0b3cae10440443/core/io/image.cpp#L2179

The lightmapper tries to create a 16384 x 16384 FORMAT_RGBAH image, but the returned size value is too big for 32 bit int and has overflowed to -2147483648. The following memset() then blows up, trying to write to invalid memory.

Lowering the LightmapGI Max Texture Size seems to help, but at least on current master it leads to some other errors, might be because of recent big rendering changes.

In general, we should try to move away from using 32 bit signed ints in data structures like these.

Calinou commented 7 months ago

Thanks for the report! Consolidating in https://github.com/godotengine/godot/issues/54679.

jcostello commented 7 months ago

@Calinou no problem. Its the same issue?