search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.35k stars 21.25k forks source link

Vulkan: Crash when baking lightmaps with LightmapGI if the resulting image is too large (due to CowData using 32-bit ints) #54679

Closed Calinou closed 4 months ago

Calinou commented 3 years ago

Godot version

4.0.dev (4651b2ae5)

System information

Fedora 34, GeForce GTX 1080 (NVIDIA 470.74)

Issue description

The editor crashes when baking lightmaps with LightmapGI in a specific scene (see the MRP included).

Backtrace:

[1] /lib64/libc.so.6(+0x3d320) [0x7fdb5cf60320] (??:0)
[2] /lib64/libc.so.6(+0x163b81) [0x7fdb5d086b81] (??:0)
[3] Image::create(int, int, bool, Image::Format) (/home/hugo/Documents/Git/godotengine/godot/core/io/image.cpp:1996)
[4] LightmapperRD::_blit_meshes_into_atlas(int, Vector<Ref<Image> >&, Vector<Ref<Image> >&, AABB&, Vector2i&, int&, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/modules/lightmapper_rd/lightmapper_rd.cpp:258)
[5] LightmapperRD::bake(Lightmapper::BakeQuality, bool, int, float, int, bool, Lightmapper::GenerateProbes, Ref<Image> const&, Basis const&, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/modules/lightmapper_rd/lightmapper_rd.cpp:684)
[6] LightmapGI::bake(Node*, String, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/scene/3d/lightmap_gi.cpp:954)
[7] LightmapGIEditorPlugin::_bake_select_file(String const&) (/home/hugo/Documents/Git/godotengine/godot/editor/plugins/lightmap_gi_editor_plugin.cpp:39)
[8] void call_with_variant_args_helper<LightmapGIEditorPlugin, String const&, 0ul>(LightmapGIEditorPlugin*, void (LightmapGIEditorPlugin::*)(String const&), Variant const**, Callable::CallError&, IndexSequence<0ul>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:223)
[9] void call_with_variant_args<LightmapGIEditorPlugin, String const&>(LightmapGIEditorPlugin*, void (LightmapGIEditorPlugin::*)(String const&), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[10] CallableCustomMethodPointer<LightmapGIEditorPlugin, String const&>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[11] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[12] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1099)
[13] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1154)
[14] EditorFileDialog::_action_pressed() (/home/hugo/Documents/Git/godotengine/godot/editor/editor_file_dialog.cpp:477)
[15] void call_with_variant_args_helper<EditorFileDialog>(EditorFileDialog*, void (EditorFileDialog::*)(), Variant const**, Callable::CallError&, IndexSequence<>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:228)
[16] void call_with_variant_args<EditorFileDialog>(EditorFileDialog*, void (EditorFileDialog::*)(), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[17] CallableCustomMethodPointer<EditorFileDialog>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[18] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[19] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1099)
[20] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1154)
[21] AcceptDialog::_ok_pressed() (/home/hugo/Documents/Git/godotengine/godot/scene/gui/dialogs.cpp:105)
[22] void call_with_variant_args_helper<AcceptDialog>(AcceptDialog*, void (AcceptDialog::*)(), Variant const**, Callable::CallError&, IndexSequence<>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:228)
[23] void call_with_variant_args<AcceptDialog>(AcceptDialog*, void (AcceptDialog::*)(), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[24] CallableCustomMethodPointer<AcceptDialog>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[25] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[26] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1099)
[27] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1154)
[28] BaseButton::_pressed() (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:126)
[29] BaseButton::on_action_event(Ref<InputEvent>) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:?)
[30] BaseButton::gui_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:67)
[31] Control::_call_gui_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/control.cpp:823)
[32] Viewport::_gui_call_input(Control*, Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:1276)
[33] Viewport::_gui_input_event(Ref<InputEvent>) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:1579)
[34] Viewport::push_input(Ref<InputEvent> const&, bool) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:2691)
[35] Window::_window_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/main/window.cpp:919)
[36] void call_with_variant_args_helper<Window, Ref<InputEvent> const&, 0ul>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, Callable::CallError&, IndexSequence<0ul>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:223)
[37] void call_with_variant_args<Window, Ref<InputEvent> const&>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[38] CallableCustomMethodPointer<Window, Ref<InputEvent> const&>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[39] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[40] DisplayServerX11::_dispatch_input_event(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:2970)
[41] DisplayServerX11::_dispatch_input_events(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:2954)
[42] Input::_parse_input_event_impl(Ref<InputEvent> const&, bool) (/home/hugo/Documents/Git/godotengine/godot/core/input/input.cpp:621)
[43] Input::flush_buffered_events() (/home/hugo/Documents/Git/godotengine/godot/core/input/input.cpp:840)
[44] DisplayServerX11::process_events() (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:3811)
[45] OS_LinuxBSD::run() (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/os_linuxbsd.cpp:338)
[46] bin/godot.linuxbsd.tools.64.llvm(main+0x1c6) [0x45c8756] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/godot_linuxbsd.cpp:58)
[47] /lib64/libc.so.6(__libc_start_main+0xd5) [0x7fdb5cf4ab75] (??:0)
[48] bin/godot.linuxbsd.tools.64.llvm(_start+0x2e) [0x45c84ce] (??:?)
-- END OF BACKTRACE --

Steps to reproduce

Minimal reproduction project

global_illumination.zip

JFonS commented 3 years ago

The atlas in this scene is 16384x16384 which means it has exactly Image::MAX_PIXELS pixels.

Since CowData uses int for almost everything, this turns into a negative value and nothing gets actually allocated.

The check in Image::create() should prevent this, but it only fails on strictly larger: https://github.com/godotengine/godot/blob/82a99951753f0fb48d4b6c99a8385009c7fb8446/core/io/image.cpp#L1987

On one hand we should make sure all the size limits are correctly checked, both on Image and LightmapGI, but I also think it would be a good time to fix CowData into using unsigned ints instead of ints, maybe even uint64_t?

Calinou commented 2 years ago

I've tried to make sure uint32_t is used everywhere in CowData, but I still get a crash at the same location. I don't get any error prints in the console before it crashes.

diff --git a/core/templates/cowdata.h b/core/templates/cowdata.h
index e79ca037db..770496d285 100644
--- a/core/templates/cowdata.h
+++ b/core/templates/cowdata.h
@@ -135,7 +135,7 @@ public:
        return _get_data();
    }

-   _FORCE_INLINE_ int size() const {
+   _FORCE_INLINE_ uint32_t size() const {
        uint32_t *size = (uint32_t *)_get_size();
        if (size) {
            return *size;
@@ -147,41 +147,41 @@ public:
    _FORCE_INLINE_ void clear() { resize(0); }
    _FORCE_INLINE_ bool is_empty() const { return _ptr == nullptr; }

-   _FORCE_INLINE_ void set(int p_index, const T &p_elem) {
+   _FORCE_INLINE_ void set(uint32_t p_index, const T &p_elem) {
        ERR_FAIL_INDEX(p_index, size());
        _copy_on_write();
        _get_data()[p_index] = p_elem;
    }

-   _FORCE_INLINE_ T &get_m(int p_index) {
+   _FORCE_INLINE_ T &get_m(uint32_t p_index) {
        CRASH_BAD_INDEX(p_index, size());
        _copy_on_write();
        return _get_data()[p_index];
    }

-   _FORCE_INLINE_ const T &get(int p_index) const {
+   _FORCE_INLINE_ const T &get(uint32_t p_index) const {
        CRASH_BAD_INDEX(p_index, size());

        return _get_data()[p_index];
    }

-   Error resize(int p_size);
+   Error resize(uint32_t p_size);

-   _FORCE_INLINE_ void remove_at(int p_index) {
+   _FORCE_INLINE_ void remove_at(uint32_t p_index) {
        ERR_FAIL_INDEX(p_index, size());
        T *p = ptrw();
-       int len = size();
-       for (int i = p_index; i < len - 1; i++) {
+       uint32_t len = size();
+       for (uint32_t i = p_index; i < len - 1; i++) {
            p[i] = p[i + 1];
        }

        resize(len - 1);
    }

-   Error insert(int p_pos, const T &p_val) {
+   Error insert(uint32_t p_pos, const T &p_val) {
        ERR_FAIL_INDEX_V(p_pos, size() + 1, ERR_INVALID_PARAMETER);
        resize(size() + 1);
-       for (int i = (size() - 1); i > p_pos; i--) {
+       for (uint32_t i = (size() - 1); i > p_pos; i--) {
            set(i, get(i - 1));
        }
        set(p_pos, p_val);
@@ -189,7 +189,7 @@ public:
        return OK;
    }

-   int find(const T &p_val, int p_from = 0) const;
+   uint32_t find(const T &p_val, uint32_t p_from = 0) const;

    _FORCE_INLINE_ CowData() {}
    _FORCE_INLINE_ ~CowData();
@@ -262,10 +262,10 @@ uint32_t CowData<T>::_copy_on_write() {
 }

 template <class T>
-Error CowData<T>::resize(int p_size) {
+Error CowData<T>::resize(uint32_t p_size) {
    ERR_FAIL_COND_V(p_size < 0, ERR_INVALID_PARAMETER);

-   int current_size = size();
+   uint32_t current_size = size();

    if (p_size == current_size) {
        return OK;
@@ -310,7 +310,7 @@ Error CowData<T>::resize(int p_size) {
        if (!__has_trivial_constructor(T)) {
            T *elems = _get_data();

-           for (int i = *_get_size(); i < p_size; i++) {
+           for (uint32_t i = *_get_size(); i < p_size; i++) {
                memnew_placement(&elems[i], T);
            }
        }
@@ -341,14 +341,14 @@ Error CowData<T>::resize(int p_size) {
 }

 template <class T>
-int CowData<T>::find(const T &p_val, int p_from) const {
-   int ret = -1;
+uint32_t CowData<T>::find(const T &p_val, uint32_t p_from) const {
+   uint32_t ret = -1;

    if (p_from < 0 || size() == 0) {
        return ret;
    }

-   for (int i = p_from; i < size(); i++) {
+   for (uint32_t i = p_from; i < size(); i++) {
        if (get(i) == p_val) {
            ret = i;
            break;
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.0.dev.custom_build (a6a2e0feb9a64a624e01789b7a3fb126e9acefc0)
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib64/libc.so.6(+0x3d320) [0x7f8b87856320] (??:0)
[2] /lib64/libc.so.6(+0x163b81) [0x7f8b8797cb81] (??:0)
[3] Image::create(int, int, bool, Image::Format) (/home/hugo/Documents/Git/godotengine/godot/core/io/image.cpp:1990)
[4] LightmapperRD::_blit_meshes_into_atlas(int, Vector<Ref<Image> >&, Vector<Ref<Image> >&, AABB&, Vector2i&, int&, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/modules/lightmapper_rd/lightmapper_rd.cpp:258)
[5] LightmapperRD::bake(Lightmapper::BakeQuality, bool, int, float, int, bool, Lightmapper::GenerateProbes, Ref<Image> const&, Basis const&, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/modules/lightmapper_rd/lightmapper_rd.cpp:684)
[6] LightmapGI::bake(Node*, String, bool (*)(float, String const&, void*, bool), void*) (/home/hugo/Documents/Git/godotengine/godot/scene/3d/lightmap_gi.cpp:954)
[7] LightmapGIEditorPlugin::_bake_select_file(String const&) (/home/hugo/Documents/Git/godotengine/godot/editor/plugins/lightmap_gi_editor_plugin.cpp:39)
[8] void call_with_variant_args_helper<LightmapGIEditorPlugin, String const&, 0ul>(LightmapGIEditorPlugin*, void (LightmapGIEditorPlugin::*)(String const&), Variant const**, Callable::CallError&, IndexSequence<0ul>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:223)
[9] void call_with_variant_args<LightmapGIEditorPlugin, String const&>(LightmapGIEditorPlugin*, void (LightmapGIEditorPlugin::*)(String const&), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[10] CallableCustomMethodPointer<LightmapGIEditorPlugin, String const&>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[11] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[12] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1104)
[13] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1159)
[14] EditorFileDialog::_action_pressed() (/home/hugo/Documents/Git/godotengine/godot/editor/editor_file_dialog.cpp:477)
[15] void call_with_variant_args_helper<EditorFileDialog>(EditorFileDialog*, void (EditorFileDialog::*)(), Variant const**, Callable::CallError&, IndexSequence<>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:228)
[16] void call_with_variant_args<EditorFileDialog>(EditorFileDialog*, void (EditorFileDialog::*)(), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[17] CallableCustomMethodPointer<EditorFileDialog>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[18] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[19] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1104)
[20] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1159)
[21] AcceptDialog::_ok_pressed() (/home/hugo/Documents/Git/godotengine/godot/scene/gui/dialogs.cpp:105)
[22] void call_with_variant_args_helper<AcceptDialog>(AcceptDialog*, void (AcceptDialog::*)(), Variant const**, Callable::CallError&, IndexSequence<>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:228)
[23] void call_with_variant_args<AcceptDialog>(AcceptDialog*, void (AcceptDialog::*)(), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[24] CallableCustomMethodPointer<AcceptDialog>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[25] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[26] Object::emit_signal(StringName const&, Variant const**, int) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1104)
[27] Object::emit_signal(StringName const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&, Variant const&) (/home/hugo/Documents/Git/godotengine/godot/core/object/object.cpp:1159)
[28] BaseButton::_pressed() (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:126)
[29] BaseButton::on_action_event(Ref<InputEvent>) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:?)
[30] BaseButton::gui_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/base_button.cpp:67)
[31] Control::_call_gui_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/gui/control.cpp:827)
[32] Viewport::_gui_call_input(Control*, Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:1277)
[33] Viewport::_gui_input_event(Ref<InputEvent>) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:1582)
[34] Viewport::push_input(Ref<InputEvent> const&, bool) (/home/hugo/Documents/Git/godotengine/godot/scene/main/viewport.cpp:2694)
[35] Window::_window_input(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/scene/main/window.cpp:923)
[36] void call_with_variant_args_helper<Window, Ref<InputEvent> const&, 0ul>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, Callable::CallError&, IndexSequence<0ul>) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:223)
[37] void call_with_variant_args<Window, Ref<InputEvent> const&>(Window*, void (Window::*)(Ref<InputEvent> const&), Variant const**, int, Callable::CallError&) (/home/hugo/Documents/Git/godotengine/godot/./core/variant/binder_common.h:338)
[38] CallableCustomMethodPointer<Window, Ref<InputEvent> const&>::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/./core/object/callable_method_pointer.h:97)
[39] Callable::call(Variant const**, int, Variant&, Callable::CallError&) const (/home/hugo/Documents/Git/godotengine/godot/core/variant/callable.cpp:51)
[40] DisplayServerX11::_dispatch_input_event(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:2970)
[41] DisplayServerX11::_dispatch_input_events(Ref<InputEvent> const&) (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:2954)
[42] Input::_parse_input_event_impl(Ref<InputEvent> const&, bool) (/home/hugo/Documents/Git/godotengine/godot/core/input/input.cpp:638)
[43] Input::flush_buffered_events() (/home/hugo/Documents/Git/godotengine/godot/core/input/input.cpp:857)
[44] DisplayServerX11::process_events() (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/display_server_x11.cpp:3818)
[45] OS_LinuxBSD::run() (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/os_linuxbsd.cpp:338)
[46] bin/godot.linuxbsd.tools.64.llvm(main+0x1c6) [0x46350f6] (/home/hugo/Documents/Git/godotengine/godot/platform/linuxbsd/godot_linuxbsd.cpp:58)
[47] /lib64/libc.so.6(__libc_start_main+0xd5) [0x7f8b87840b75] (??:0)
[48] bin/godot.linuxbsd.tools.64.llvm(_start+0x2e) [0x4634e6e] (??:?)
-- END OF BACKTRACE --
================================================================
[1]    22083 IOT instruction (core dumped)  bin/godot.linuxbsd.tools.64.llvm
bruvzg commented 2 years ago

It seems to be crashing (at least on macOS) at https://github.com/godotengine/godot/blob/9c6c71bbdc42c42a22bed9fd536da8929e9ad0ec/core/io/image.cpp#L1960 size is int and is overflown, value at the moment of crash is -2147483648.

jitspoe commented 2 years ago

Not sure if this is the same crash, but I'm trying to light something imported from QODOT.

    godot.windows.opt.tools.64.exe!memcpy() Line 433    Unknown
>   godot.windows.opt.tools.64.exe!RenderingDeviceVulkan::texture_get_data(RID p_texture, unsigned int p_layer) Line 2806   C++
    godot.windows.opt.tools.64.exe!RendererSceneRenderRD::bake_render_uv2(RID p_base, const Vector<RID> & p_material_overrides, const Vector2i & p_image_size) Line 5689    C++
    godot.windows.opt.tools.64.exe!RendererSceneCull::bake_render_uv2(RID p_base, const Vector<RID> & p_material_overrides, const Vector2i & p_image_size) Line 3969    C++
    godot.windows.opt.tools.64.exe!RenderingServerDefault::bake_render_uv2(RID p1, const Vector<RID> & p2, const Vector2i & p3) Line 772    C++
    godot.windows.opt.tools.64.exe!LightmapGI::bake(Node * p_from_node, String p_image_data_path, bool(*)(float, const String &, void *, bool) p_bake_step, void * p_bake_userdata) Line 763    C++
    godot.windows.opt.tools.64.exe!LightmapGIEditorPlugin::_bake_select_file(const String & p_file) Line 46 C++
    [Inline Frame] godot.windows.opt.tools.64.exe!call_with_variant_args_helper(LightmapGIEditorPlugin *) Line 236  C++
    [Inline Frame] godot.windows.opt.tools.64.exe!call_with_variant_args(LightmapGIEditorPlugin * p_instance, void(LightmapGIEditorPlugin::*)(const String &) p_method, const Variant * *) Line 350 C++
    godot.windows.opt.tools.64.exe!CallableCustomMethodPointer<LightmapGIEditorPlugin,String const &>::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 104   C++
    godot.windows.opt.tools.64.exe!Callable::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 51  C++
    godot.windows.opt.tools.64.exe!Object::emit_signalp(const StringName & p_name, const Variant * * p_args, int p_argcount) Line 965   C++
    godot.windows.opt.tools.64.exe!Object::emit_signal<String>(const StringName & p_name, String <p_args_0>) Line 813   C++
    godot.windows.opt.tools.64.exe!EditorFileDialog::_action_pressed() Line 487 C++
    godot.windows.opt.tools.64.exe!Callable::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 51  C++
    godot.windows.opt.tools.64.exe!Object::emit_signalp(const StringName & p_name, const Variant * * p_args, int p_argcount) Line 965   C++
    [Inline Frame] godot.windows.opt.tools.64.exe!Object::emit_signal(const StringName &) Line 813  C++
    godot.windows.opt.tools.64.exe!AcceptDialog::_ok_pressed() Line 107 C++
    godot.windows.opt.tools.64.exe!Callable::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 51  C++
    godot.windows.opt.tools.64.exe!Object::emit_signalp(const StringName & p_name, const Variant * * p_args, int p_argcount) Line 965   C++
    [Inline Frame] godot.windows.opt.tools.64.exe!Object::emit_signal(const StringName &) Line 813  C++
    godot.windows.opt.tools.64.exe!BaseButton::_pressed() Line 134  C++
    godot.windows.opt.tools.64.exe!BaseButton::on_action_event(Ref<InputEvent> p_event) Line 176    C++
    godot.windows.opt.tools.64.exe!BaseButton::gui_input(const Ref<InputEvent> & p_event) Line 67   C++
    godot.windows.opt.tools.64.exe!Control::_call_gui_input(const Ref<InputEvent> & p_event) Line 944   C++
    godot.windows.opt.tools.64.exe!Viewport::_gui_call_input(Control * p_control, const Ref<InputEvent> & p_input) Line 1281    C++
    godot.windows.opt.tools.64.exe!Viewport::_gui_input_event(Ref<InputEvent> p_event) Line 1589    C++
    godot.windows.opt.tools.64.exe!Viewport::push_input(const Ref<InputEvent> & p_event, bool p_local_coords) Line 2721 C++
    godot.windows.opt.tools.64.exe!Window::_window_input(const Ref<InputEvent> & p_ev) Line 1005    C++
    [Inline Frame] godot.windows.opt.tools.64.exe!call_with_variant_args_helper(Window *) Line 236  C++
    [Inline Frame] godot.windows.opt.tools.64.exe!call_with_variant_args(Window * p_instance, void(Window::*)(const Ref<InputEvent> &) p_method, const Variant * *) Line 350    C++
    godot.windows.opt.tools.64.exe!CallableCustomMethodPointer<Window,Ref<InputEvent> const &>::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 104  C++
    godot.windows.opt.tools.64.exe!Callable::call(const Variant * * p_arguments, int p_argcount, Variant & r_return_value, Callable::CallError & r_call_error) Line 51  C++
    godot.windows.opt.tools.64.exe!DisplayServerWindows::_dispatch_input_event(const Ref<InputEvent> & p_event) Line 2077   C++
    godot.windows.opt.tools.64.exe!Input::_parse_input_event_impl(const Ref<InputEvent> & p_event, bool p_is_emulated) Line 662 C++
    godot.windows.opt.tools.64.exe!Input::flush_buffered_events() Line 888  C++
    godot.windows.opt.tools.64.exe!DisplayServerWindows::process_events() Line 1784 C++
    godot.windows.opt.tools.64.exe!OS_Windows::run() Line 777   C++
    godot.windows.opt.tools.64.exe!widechar_main(int argc, wchar_t * * argv) Line 175   C++
    godot.windows.opt.tools.64.exe!_main() Line 199 C++
    godot.windows.opt.tools.64.exe!main(int argc, char * * argv) Line 211   C++
    [External Code] 
        Vector<uint8_t> buffer_data;
        {
            buffer_data.resize(buffer_size);
            uint8_t *w = buffer_data.ptrw();
            memcpy(w, buffer_mem, buffer_size);
        }

        vmaUnmapMemory(allocator, tmp_buffer.allocation);

buffer_size = 2224175008 (slightly larger than 2 gigs?)

I tried setting the max lightmap size to 2048, the smallest it would allow, but I still get the crash.

Edit: Changing the texel unwrap size setting in Qodot from 1 to 16 seems to have fixed it.