search

baldurk / renderdoc

RenderDoc is a stand-alone graphics debugging tool.
https://renderdoc.org
MIT License
8.92k stars 1.34k forks source link

Mesh viewer vertex selection is rendered in the wrong location #2386

Closed Firestar99 closed 2 years ago

Firestar99 commented 3 years ago

Description

In the mesh viewer the vertex selection rendering seems to be broken. You can select the vertex just fine with left-click on the rendered model (and from the table), however the overlay displaying the selected vertex via the blue box as well as the red triangle to which the vertex is associated to does not render in the correct location. The red triangle may also appear squished or pulled apart suggesting non-linear scaling happening, as if a different projection matrix was used. My Project likely uses an infinite zFar plane projection matrix, I am not too familiar with this but can ask around more if needed.

image

Steps to reproduce

  1. create and load a capture of a DX11 application (not sure if I can share a capture) ** may require an infinite zFar plane projection matrix to be used
  2. load the capture and select any 3D draw call (using said projection matrix, if necessary)
  3. go to mesh viewer and observe result

Environment

Firestar99 commented 3 years ago

Sorry for previously not using the template, github seems to have a bug where if you press the new issue button and are not yet logged in it will redirect you to an empty issue without a template or template selection. :)

baldurk commented 3 years ago

No worries, I'm sorry that github has bugs like that where you can bypass the templates which are otherwise set up as completely required - it does make for a crappy user experience. Sadly I can't tell a case like that from someone who deleted/ignored the template and they're required for good reason.

Can you share a reproducible test case that shows this problem? If you can't share the capture publicly you can also email it to me privately, or else there are DX11 demos that use different projection methods including infinite zFar so if you believe it's caused by that then testing/modifiying a public demo would do the trick. If the problem really is as completely general as "any draw on any DX11 program" then a reproducible test case doesn't do any harm in identifying a specific instance where it fails, but in the majority of cases with bugs it tends to be something more specific that is hard to recreate from a very general high-level description.

Alternatively if this did regress between v1.15 and v1.16 you could use the nightly builds to narrow down which commit(s) caused it to break which would also be useful. It depends what you have time for.

zrkn commented 2 years ago

I'm having same problem after v1.16 upgrade. You can reproduce it by capturing 08. Drawing a Cube demo from BeginnerDirect3D11. изображение

baldurk commented 2 years ago

Thanks for the repro case. This looks like it's working on latest code so it's possibly something I've fixed already, you can double check with a recent nightly build and see if it works for you.

This issue was about to be closed for inactivity, so I'll leave it open until then to see if the OP confirms that this is the same bug or a different one,

zrkn commented 2 years ago

Yep, works fine on 2021-10-31 nightly for me. Thanks!

Firestar99 commented 2 years ago

I can confirm too that in 2021-11-01 20b12574 the issue is resolved, thanks!

Sorry for the long inactivity, I had to take a break from work to finish the writing part of my bachelor thesis :)