search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
88.65k stars 20.1k forks source link

AddressSanitizer: heap-buffer-overflow caused by embree buffer padding added using resize() instead of reserve() #94548

Open jamie-pate opened 1 month ago

jamie-pate commented 1 month ago

Tested versions

Reproducible in current HEAD: ff8a2780ee777c2456ce42368e1065774c7c4c3f

System information

Godot Engine v4.3.beta.custom_build.eb5f1299b (2024-07-19 02:22:03 UTC) - https://godotengine.org Vulkan 1.3.242 - Forward+ - Using Device #2: NVIDIA - NVIDIA GeForce RTX 3050 Laptop GPU

Issue description

==592286==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61b00011a990 at pc 0x55555994a0b5 bp 0x7fffffff8d50 sp 0x7fffffff8d40
READ of size 4 at 0x61b00011a990 thread T0
    #0 0x55555994a0b4 in embree::RawBufferView::checkPadding16() const thirdparty/embree/kernels/common/buffer.h:215
    #1 0x55555994a0b4 in embree::TriangleMesh::setBuffer(RTCBufferType, unsigned int, RTCFormat, embree::Ref<embree::Buffer> const&, unsigned long, unsigned long, unsigned int) thirdparty/embree/kernels/common/scene_triangle_mesh.cpp:54
    #2 0x55555991d0d9 in rtcSetSharedGeometryBuffer thirdparty/embree/kernels/common/rtcore.cpp:1670
    #3 0x555559fd6575 in RaycastOcclusionCull::Scenario::update() modules/raycast/raycast_occlusion_cull.cpp:483
    #4 0x555559fd97d5 in RaycastOcclusionCull::buffer_update(RID, Transform3D const&, Projection const&, bool) modules/raycast/raycast_occlusion_cull.cpp:616
    #5 0x555566c6dc4b in RendererSceneCull::render_camera(Ref<RenderSceneBuffers> const&, RID, RID, RID, Vector2, unsigned int, float, RID, Ref<XRInterface>&, RenderingMethod::RenderInfo*) servers/rendering/renderer_scene_cull.cpp:2644
    #6 0x555566d29b7e in RendererViewport::_draw_3d(RendererViewport::Viewport*) servers/rendering/renderer_viewport.cpp:248
    #7 0x555566d2b690 in RendererViewport::_draw_viewport(RendererViewport::Viewport*) servers/rendering/renderer_viewport.cpp:314
    #8 0x555566d3e26d in RendererViewport::draw_viewports(bool) servers/rendering/renderer_viewport.cpp:808
    #9 0x5555656015a4 in RenderingServerDefault::_draw(bool, double) servers/rendering/rendering_server_default.cpp:87
    #10 0x555565608d47 in RenderingServerDefault::draw(bool, double) servers/rendering/rendering_server_default.cpp:410
    #11 0x55555707db84 in Main::iteration() main/main.cpp:4118
    #12 0x555556ea86a9 in OS_LinuxBSD::run() platform/linuxbsd/os_linuxbsd.cpp:962
    #13 0x555556e8e776 in main platform/linuxbsd/godot_linuxbsd.cpp:85
    #14 0x7ffff7029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #15 0x7ffff7029e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #16 0x555556e8e364 in _start (/home/jpate/build/src/godot/bin/godot.linuxbsd.editor.x86_64.san+0x193a364)

0x61b00011a990 is located 0 bytes to the right of 1552-byte region [0x61b00011a380,0x61b00011a990)
allocated by thread T0 here:
    #0 0x7ffff74b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x555568a46fe6 in Memory::alloc_static(unsigned long, bool) core/os/memory.cpp:75
    #2 0x555568a4744b in Memory::realloc_static(void*, unsigned long, bool) core/os/memory.cpp:99
    #3 0x555558b4f757 in LocalVector<Vector3, unsigned int, false, false>::resize(unsigned int) core/templates/local_vector.h:161
    #4 0x555559fd33e3 in RaycastOcclusionCull::Scenario::_update_dirty_instance(int, RID*) modules/raycast/raycast_occlusion_cull.cpp:359
    #5 0x555559fd57f7 in RaycastOcclusionCull::Scenario::update() modules/raycast/raycast_occlusion_cull.cpp:452
    #6 0x555559fd97d5 in RaycastOcclusionCull::buffer_update(RID, Transform3D const&, Projection const&, bool) modules/raycast/raycast_occlusion_cull.cpp:616
    #7 0x555566c6dc4b in RendererSceneCull::render_camera(Ref<RenderSceneBuffers> const&, RID, RID, RID, Vector2, unsigned int, float, RID, Ref<XRInterface>&, RenderingMethod::RenderInfo*) servers/rendering/renderer_scene_cull.cpp:2644
    #8 0x555566d29b7e in RendererViewport::_draw_3d(RendererViewport::Viewport*) servers/rendering/renderer_viewport.cpp:248
    #9 0x555566d2b690 in RendererViewport::_draw_viewport(RendererViewport::Viewport*) servers/rendering/renderer_viewport.cpp:314
    #10 0x555566d3e26d in RendererViewport::draw_viewports(bool) servers/rendering/renderer_viewport.cpp:808
    #11 0x5555656015a4 in RenderingServerDefault::_draw(bool, double) servers/rendering/rendering_server_default.cpp:87
    #12 0x555565608d47 in RenderingServerDefault::draw(bool, double) servers/rendering/rendering_server_default.cpp:410
    #13 0x55555707db84 in Main::iteration() main/main.cpp:4118
    #14 0x555556ea86a9 in OS_LinuxBSD::run() platform/linuxbsd/os_linuxbsd.cpp:962
    #15 0x555556e8e776 in main platform/linuxbsd/godot_linuxbsd.cpp:85
    #16 0x7ffff7029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: heap-buffer-overflow thirdparty/embree/kernels/common/buffer.h:215 in embree::RawBufferView::checkPadding16() const
Shadow bytes around the buggy address:
  0x0c368001b4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b4f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c368001b530: 00 00[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c368001b540: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c368001b550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c368001b580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==592286==ABORTING

Steps to reproduce

Enable asan and run a scene that uses occlusion culling? It only happens under certain conditions, even with asan enabled.

Minimal reproduction project (MRP)

asan-embree-output.txt

jamie-pate commented 1 month ago

An alternate solution would involve upstreaming this fix to embree I suppose, since their buffer check calculation dereferencing at 24 bytes instead of 16 is the actual cause.. (due to the size of int* changing on x86_64 vs x86_32)

(upstream issue reported https://github.com/RenderKit/embree/issues/495)