search

baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
9.02k stars 1.35k forks source link

Crash while loading replay #3405

Closed iLikeTrioxin closed 3 months ago

iLikeTrioxin commented 3 months ago

Description

Hi, upfront sorry for any mistake but this is my first time opening an issue. I wanted to debug my game that previously worked in renderdoc but it crashed while loading the capture (V1.34).

I had renderdoc built from source(V1.33) laying around so i checked with gdb:

_mesa_get_uniform () at ../mesa-24.1.5/src/mesa/main/uniform_query.cpp:376
376         src = (gl_constant_value *) uni->driver_storage[0].data +
(gdb) bt
#0  _mesa_get_uniform () at ../mesa-24.1.5/src/mesa/main/uniform_query.cpp:376
#1  0x00007ffff58076d1 in GetCurrentBinding (curProg=289, refl=0x7ffef80429b0, resource=..., slot=@0x7fffa95f5954: 0, used=@0x7fffa95f594c: false)
    at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_shader_refl.cpp:2490
#2  0x00007ffff573a2fe in WrappedOpenGL::AddUsage (this=0x7fff44172de0, a=...) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:5552
#3  0x00007ffff573b1ec in WrappedOpenGL::AddAction (this=0x7fff44172de0, a=...) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:5737
#4  0x00007ffff5899f6d in WrappedOpenGL::Serialise_glDrawElements<ReadSerialiser> (this=0x7fff44172de0, ser=..., mode=eGL_TRIANGLES, count=1056, type=eGL_UNSIGNED_INT, indicesPtr=0x0)
    at /home/samuel/beta/renderdoc/renderdoc/driver/gl/wrappers/gl_draw_funcs.cpp:1466
#5  0x00007ffff5735387 in WrappedOpenGL::ProcessChunk (this=0x7fff44172de0, ser=..., chunk=GLChunk::glDrawElements) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:3965
#6  0x00007ffff5739588 in WrappedOpenGL::ContextProcessChunk (this=0x7fff44172de0, ser=..., chunk=GLChunk::glDrawElements) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:5361
#7  0x00007ffff5738d71 in WrappedOpenGL::ContextReplayLog (this=0x7fff44172de0, readType=CaptureState::LoadingReplaying, startEventID=0, endEventID=0, partial=false)
    at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:5272
#8  0x00007ffff57335e2 in WrappedOpenGL::ReadLogInitialisation (this=0x7fff44172de0, rdc=0x7fff44001470, storeStructuredBuffers=false) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_driver.cpp:3487
#9  0x00007ffff57b9e7f in GLReplay::ReadLogInitialisation (this=0x7fff44179280, rdc=0x7fff44001470, storeStructuredBuffers=false) at /home/samuel/beta/renderdoc/renderdoc/driver/gl/gl_replay.cpp:118
#10 0x00007ffff6bad725 in ReplayController::PostCreateInit (this=0x7fff44002bb0, device=0x7fff44179280, rdc=0x7fff44001470) at /home/samuel/beta/renderdoc/renderdoc/replay/replay_controller.cpp:2204
#11 0x00007ffff6bacf18 in ReplayController::CreateDevice (this=0x7fff44002bb0, rdc=0x7fff44001470, opts=...) at /home/samuel/beta/renderdoc/renderdoc/replay/replay_controller.cpp:2171
#12 0x00007ffff6b7a6c8 in CaptureFile::OpenCapture (this=0x7fff44001300, opts=..., progress=...) at /home/samuel/beta/renderdoc/renderdoc/replay/capture_file.cpp:371
#13 0x0000555555c14a06 in ReplayManager::run(int, QString const&, ReplayOptions const&, std::function<void (float)>) ()
#14 0x0000555555c14b00 in std::_Function_handler<void (), ReplayManager::OpenCapture(QString const&, ReplayOptions const&, std::function<void (float)>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) [clone .lto_priv.0] ()
#15 0x0000555555bfa7a6 in LambdaThread::process() ()
#16 0x00007ffff42df6ee in QtPrivate::QSlotObjectBase::call (this=0x7fffbc0023e0, r=0x7fffbc002cb0, a=0x7fffa95ffcd0, this=<optimized out>, r=<optimized out>, a=<optimized out>)
    at ../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#17 doActivate<false> (sender=0x7fffbc042f50, signal_index=3, argv=0x7fffa95ffcd0) at kernel/qobject.cpp:3925
#18 0x00007ffff40ee18e in QThread::started (this=this@entry=0x7fffbc042f50, _t1=...) at .moc/moc_qthread.cpp:163
#19 0x00007ffff40f2602 in operator() (__closure=<optimized out>) at thread/qthread_unix.cpp:345
#20 (anonymous namespace)::terminate_on_exception<QThreadPrivate::start(void*)::<lambda()> > (t=<optimized out>) at thread/qthread_unix.cpp:287
#21 QThreadPrivate::start (arg=0x7fffbc042f50) at thread/qthread_unix.cpp:310
#22 0x00007ffff3aa339d in start_thread (arg=<optimized out>) at pthread_create.c:447
#23 0x00007ffff3b2849c in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

As it turned out the problem was vertex shader for animated character I recently added:

#version 450 core

const int MAX_BONES = 200;
const int MAX_BONE_INFLUENCE = 4;

layout (location = 0) in vec3 pos;
layout (location = 1) in vec2 tex;
layout (location = 2) in vec3 norm;
layout(location = 3) in ivec4 boneID; 
layout(location = 4) in vec4 boneWeight;

layout (location = 0) out vec3 _FragPos;
layout (location = 1) out vec3 _Normal;
layout (location = 2) out vec2 _TexCoords;

layout (binding = 0) uniform View{
    mat4 model;
    mat4 projection;
    mat4 view;
} v;

layout (binding = 1) uniform Anim{
    mat4 finalBonesMatrices[MAX_BONES];
} ani;

void main() {
    mat4 BoneTransform = mat4(1.0f);
    BoneTransform += ani.finalBonesMatrices[boneID[0]] * boneWeight[0];
    BoneTransform += ani.finalBonesMatrices[boneID[1]] * boneWeight[1];
    BoneTransform += ani.finalBonesMatrices[boneID[2]] * boneWeight[2];
    BoneTransform += ani.finalBonesMatrices[boneID[3]] * boneWeight[3];

    vec4 totalPositionX = BoneTransform * vec4(pos, 1.0);

    mat4 viewModel = v.view * v.model;
    _FragPos = vec3(v.model * totalPositionX);
    gl_Position = v.projection * viewModel * totalPositionX;
    _Normal  = (v.model * vec4(BoneTransform * vec4(norm, 0.0))).xyz;
    _TexCoords = tex;
}

It is because of second UBO "ani", when i comment out 4 lines adding together bone transformation it loads just fine. I went into pipeline view and saw that UBO "ani" at binding id 1 has correct data so I dont know why it crashes (even if it was wrong it still shouldn't crash, right?).

Steps to reproduce

BrokenCapture: https://drive.google.com/file/d/1kZ0nrAUr8qWwlMTJrkus7gQU6j0iWCHm/view?usp=sharing WorkingCapture: https://drive.google.com/file/d/1H4sr9DXdOix2XQmsdzZyQZ2KOJuxKp26/view?usp=sharing

Environment

baldurk commented 3 months ago

That commit should solve the issue, although it shouldn't have crashed (that seems to be missing validation on mesa's part since for GL in theory invalid calls should error) RenderDoc was incorrectly querying for a uniform's value with a binding instead of a location.