search

google / model-viewer

Easily display interactive 3D models on the web and in AR!
https://modelviewer.dev
Apache License 2.0
6.91k stars 816 forks source link

Strange display of roughness in model-viewer #3145

Closed superLeaAtWork closed 2 years ago

superLeaAtWork commented 2 years ago

Description

I viewed a model in the model-viewer editor: https://modelviewer.dev/editor/ The model has a not so rough surface. The surface appeared very pixelated and somehow broken. When I removed the roughness texture in the viewer, the model appears correct. The I tried a model that worked before - and also this model is broken now.

When I select a predefined model like the water bottles model - all parts appear also broken. See the example images below.

Live Demo

https://glitch.com/edit/#!/model-viewer

image image

Version

Browser Affected

Chrome

AR

elalish commented 2 years ago

I can't repro; can you please fill in all the details of the device/OS/browser versions you're using to test? This has the look of a GPU driver bug.

felipe-chamas commented 2 years ago

I can confirm the behaviour as well

1.9.2

1 9 2

1.10.1

1 10 1

Specs

PC

image

Chrome

Version 97.0.4692.99 (64 bit)

Firefox

Version 97.0b7 (64 bit)

elalish commented 2 years ago

@FelipeR2U Can I assume this is a Windows machine? @mrdoob This looks like a regression somewhere in the last few versions for three.js. This noise looks very much like a driver bug to me (especially since it doesn't repro on Mac or mobile), but I guess we must have triggered it somehow? Have you seen any similar issues in the three.js repo?

I didn't even notice the difference at first on FelipeR2U's screenshots, but there is a band of gray noise going up the middle of the sleeve. Way worse up on the bottles.

mrdoob commented 2 years ago

I suspect it is related to the RGBEEncoding/RGBEFormat removal... https://www.facebook.com/groups/threejs/permalink/1861902697330383/

elalish commented 2 years ago

Hmm, so maybe a HalfFloat bug on some Windows GPU driver? Ugh. Any chance you could get kbr to take a look?

mrdoob commented 2 years ago

/ping @kenrussell

kenrussell commented 2 years ago

I can't reproduce on an NVIDIA Quadro K2200. about:gpu from Chrome on this system: https://pastebin.com/PrtQanE3

Please gather about:gpu from an affected system and link to it here; we'll file a bug on the Chromium issue tracker to investigate in more detail. Please provide clear and concise reproduction steps. I went to https://modelviewer.dev/editor/ , and selected the "Water Bottles" model. If that page is likely to evolve then please provide a stable reproduction somewhere.

Thanks.

mrdoob commented 2 years ago

@superLeaAtWork @FelipeR2U

Can you share screenshots of these two examples?

https://modelviewer.dev/examples/lightingandenv/#renderExposure https://modelviewer.dev/examples/scenegraph/#changeMaterial

kenrussell commented 2 years ago

Note: still unable to reproduce the failure after upgrading to the latest supported feature driver for this GPU, 496.49. https://pastebin.com/e8WPsmyA

It's likely specific to the GPU family.

These kinds of twinkling artifacts look to me like undefined behavior in GLSL like calling pow() with a negative exponent. Could this be investigated with an eye toward that in the associated shaders?

superLeaAtWork commented 2 years ago

Hello, @mrdoob hiere the two examples:

image

image

I tested it on 3 different windows machines. I tested in on chrome (Version 97.0.4692.99 (Offizieller Build) (64-Bit)) and fireforx (96.0.2 (64-Bit)). The graphic-card on my machine is NVIDIA GeForce RTX 2070 SUPER. my machine works with Windows 10 Pro. The processor is a AMD Ryzen 9 3900XT 12-Core Processor (64Bit).

The other machines have a different setup. But always windows PC.

felipe-chamas commented 2 years ago

@elalish , yes, it's a Windows 11 machine This gray noise is hard to notice on the screenshot but very noticeable while interacting with the model.

@mrdoob , I have the same results as superLeaAtWork. It appears to have some joints on the sphere

felipe-chamas commented 2 years ago

@kenrussell

Graphics Feature Status

  • Canvas: Hardware accelerated
  • Canvas out-of-process rasterization: Disabled
  • Compositing: Hardware accelerated
  • Multiple Raster Threads: Enabled
  • Out-of-process Rasterization: Hardware accelerated
  • OpenGL: Enabled
  • Rasterization: Hardware accelerated
  • Raw Draw: Disabled
  • Skia Renderer: Enabled
  • Video Decode: Hardware accelerated
  • Video Encode: Hardware accelerated
  • Vulkan: Disabled
  • WebGL: Hardware accelerated
  • WebGL2: Hardware accelerated

Driver Bug Workarounds

  • clear_uniforms_before_first_program_use
  • decode_encode_srgb_for_generatemipmap
  • disable_delayed_copy_nv12
  • enable_webgl_timer_query_extensions
  • exit_on_context_lost
  • disabled_extension_GL_KHR_blend_equation_advanced
  • disabled_extension_GL_KHR_blend_equation_advanced_coherent

Problems Detected

ANGLE Features

  • allow_compressed_formats (Frontend workarounds): Enabled: true
    Allow compressed formats
  • disable_anisotropic_filtering (Frontend workarounds): Disabled
    Disable support for anisotropic filtering
  • disable_program_binary (Frontend features) anglebug:5007: Disabled
    Disable support for GL_OES_get_program_binary
  • disable_program_caching_for_transform_feedback (Frontend workarounds): Disabled
    On some GPUs, program binaries don't contain transform feedback varyings
  • enableCompressingPipelineCacheInThreadPool (Frontend workarounds) anglebug:4722: Disabled: false
    Enable compressing pipeline cache in thread pool.
  • enableProgramBinaryForCapture (Frontend features) anglebug:5658: Disabled
    Even if FrameCapture is enabled, enable GL_OES_get_program_binary
  • enable_capture_limits (Frontend features) anglebug:5750: Disabled
    Set the context limits like frame capturing was enabled
  • forceInitShaderVariables (Frontend features): Disabled
    Force-enable shader variable initialization
  • forceRobustResourceInit (Frontend features) anglebug:6041: Disabled
    Force-enable robust resource init
  • lose_context_on_out_of_memory (Frontend workarounds): Enabled: true
    Some users rely on a lost context notification if a GL_OUT_OF_MEMORY error occurs
  • scalarize_vec_and_mat_constructor_args (Frontend workarounds) 1165751: Disabled: false
    Always rewrite vec/mat constructors to be consistent
  • add_mock_texture_no_render_target (D3D workarounds) anglebug:2152: Disabled: isIntel && capsVersion >= IntelDriverVersion(160000) && capsVersion < IntelDriverVersion(164815)
    On some drivers when rendering with no render target, two bugs lead to incorrect behavior
  • allowES3OnFL10_0 (D3D workarounds): Disabled: false
    Allow ES3 on 10.0 devices
  • allow_clear_for_robust_resource_init (D3D workarounds) 941620: Enabled: true
    Some drivers corrupt texture data when clearing for robust resource initialization.
  • allow_translate_uniform_block_to_structured_buffer (D3D workarounds) anglebug:3682: Enabled: IsWin10OrGreater()
    There is a slow fxc compile performance issue with dynamic uniform indexing if translating a uniform block with a large array member to cbuffer.
  • call_clear_twice (D3D workarounds) 655534: Disabled: isIntel && isSkylake && capsVersion >= IntelDriverVersion(160000) && capsVersion < IntelDriverVersion(164771)
    Using clear() may not take effect
  • depth_stencil_blit_extra_copy (D3D workarounds) anglebug:1452: Disabled: (part1 <= 13u && part2 < 6881) && isNvidia && driverVersionValid
    Bug in some drivers triggers a TDR when using CopySubresourceRegion from a staging texture to a depth/stencil
  • disable_b5g6r5_support (D3D workarounds): Disabled: (isIntel && capsVersion >= IntelDriverVersion(150000) && capsVersion < IntelDriverVersion(154539)) || isAMD
    Textures with the format DXGI_FORMAT_B5G6R5_UNORM have incorrect data
  • emulate_isnan_float (D3D workarounds) 650547: Disabled: isIntel && isSkylake && capsVersion >= IntelDriverVersion(160000) && capsVersion < IntelDriverVersion(164542)
    Using isnan() on highp float will get wrong answer
  • emulate_tiny_stencil_textures (D3D workarounds): Disabled: isAMD && !(deviceCaps.featureLevel < D3D_FEATURE_LEVEL_10_1)
    1x1 and 2x2 mips of depth/stencil textures aren't sampled correctly
  • expand_integer_pow_expressions (D3D workarounds): Enabled: true
    The HLSL optimizer has a bug with optimizing 'pow' in certain integer-valued expressions
  • flush_after_ending_transform_feedback (D3D workarounds): Enabled: isNvidia
    Some drivers sometimes write out-of-order results to StreamOut buffers when transform feedback is used to repeatedly write to the same buffer positions
  • force_atomic_value_resolution (D3D workarounds) anglebug:3246: Enabled: isNvidia
    On some drivers the return value from RWByteAddressBuffer.InterlockedAdd does not resolve when used in the .yzw components of a RWByteAddressBuffer.Store operation
  • get_dimensions_ignores_base_level (D3D workarounds): Enabled: isNvidia
    Some drivers do not take into account the base level of the texture in the results of the HLSL GetDimensions builtin
  • mrt_perf_workaround (D3D workarounds): Enabled: true
    Some drivers have a bug where they ignore null render targets
  • pre_add_texel_fetch_offsets (D3D workarounds): Disabled: isIntel
    HLSL's function texture.Load returns 0 when the parameter Location is negative, even if the sum of Offset and Location is in range
  • rewrite_unary_minus_operator (D3D workarounds): Disabled: isIntel && (isBroadwell || isHaswell) && capsVersion >= IntelDriverVersion(150000) && capsVersion < IntelDriverVersion(154624)
    Evaluating unary minus operator on integer may get wrong answer in vertex shaders
  • select_view_in_geometry_shader (D3D workarounds): Disabled: !deviceCaps.supportsVpRtIndexWriteFromVertexShader
    The viewport or render target slice will be selected in the geometry shader stage for the ANGLE_multiview extension
  • set_data_faster_than_image_upload (D3D workarounds): Enabled: !(isIvyBridge || isBroadwell || isHaswell)
    Set data faster than image upload
  • skip_vs_constant_register_zero (D3D workarounds): Enabled: isNvidia
    In specific cases the driver doesn't handle constant register zero correctly
  • use_instanced_point_sprite_emulation (D3D workarounds): Disabled: isFeatureLevel9_3
    Some D3D11 renderers do not support geometry shaders for pointsprite emulation
  • use_system_memory_for_constant_buffers (D3D workarounds) 593024: Disabled: isIntel
    Copying from staging storage to constant buffer storage does not work
  • zero_max_lod (D3D workarounds): Disabled: isFeatureLevel9_3
    Missing an option to disable mipmaps on a mipmapped texture

DAWN Info


    <Discrete GPU> D3D12 backend - NVIDIA GeForce RTX 2070
    [Default Toggle Names]
  • lazy_clear_resource_on_first_use: https://crbug.com/dawn/145: Clears resource to zero on first usage. This initializes the resource so that no dirty bits from recycled memory is present in the new resource.
  • use_d3d12_resource_heap_tier2: https://crbug.com/dawn/27: Enable support for resource heap tier 2. Resource heap tier 2 allows mixing of texture and buffers in the same heap. This allows better heap re-use and reduces fragmentation.
  • use_d3d12_render_pass: https://crbug.com/dawn/36: Use the D3D12 render pass API introduced in Windows build 1809 by default. On versions of Windows prior to build 1809, or when this toggle is turned off, Dawn will emulate a render pass.
  • use_d3d12_residency_management: https://crbug.com/dawn/193: Enable residency management. This allows page-in and page-out of resource heaps in GPU memory. This component improves overcommitted performance by keeping the most recently used resources local to the GPU. Turning this component off can cause allocation failures when application memory exceeds physical device memory.
  • disallow_unsafe_apis: http://crbug.com/1138528: Produces validation errors on API entry points or parameter combinations that aren't considered secure yet.
    • [WebGPU Forced Toggles - enabled]
  • disallow_spirv: https://crbug.com/1214923: Disallow usage of SPIR-V completely so that only WGSL is used for shader modules.This is useful to prevent a Chromium renderer process from successfully sendingSPIR-V code to be compiled in the GPU process.
    • [Supported Features]
  • texture-compression-bc
  • pipeline-statistics-query
  • timestamp-query
  • dawn-internal-usages
  • multiplanar-formats

    • <CPU> D3D12 backend - Microsoft Basic Render Driver
    [Default Toggle Names]
  • lazy_clear_resource_on_first_use: https://crbug.com/dawn/145: Clears resource to zero on first usage. This initializes the resource so that no dirty bits from recycled memory is present in the new resource.
  • use_d3d12_resource_heap_tier2: https://crbug.com/dawn/27: Enable support for resource heap tier 2. Resource heap tier 2 allows mixing of texture and buffers in the same heap. This allows better heap re-use and reduces fragmentation.
  • use_d3d12_render_pass: https://crbug.com/dawn/36: Use the D3D12 render pass API introduced in Windows build 1809 by default. On versions of Windows prior to build 1809, or when this toggle is turned off, Dawn will emulate a render pass.
  • use_d3d12_residency_management: https://crbug.com/dawn/193: Enable residency management. This allows page-in and page-out of resource heaps in GPU memory. This component improves overcommitted performance by keeping the most recently used resources local to the GPU. Turning this component off can cause allocation failures when application memory exceeds physical device memory.
  • disallow_unsafe_apis: http://crbug.com/1138528: Produces validation errors on API entry points or parameter combinations that aren't considered secure yet.
    • [WebGPU Forced Toggles - enabled]
  • disallow_spirv: https://crbug.com/1214923: Disallow usage of SPIR-V completely so that only WGSL is used for shader modules.This is useful to prevent a Chromium renderer process from successfully sendingSPIR-V code to be compiled in the GPU process.
    • [Supported Features]
  • texture-compression-bc
  • pipeline-statistics-query
  • timestamp-query
  • dawn-internal-usages
  • multiplanar-formats

    • <Discrete GPU> Vulkan backend - NVIDIA GeForce RTX 2070
    [Default Toggle Names]
  • lazy_clear_resource_on_first_use: https://crbug.com/dawn/145: Clears resource to zero on first usage. This initializes the resource so that no dirty bits from recycled memory is present in the new resource.
  • use_temporary_buffer_in_texture_to_texture_copy: https://crbug.com/dawn/42: Split texture-to-texture copy into two copies: copy from source texture into a temporary buffer, and copy from the temporary buffer into the destination texture when copying between compressed textures that don't have block-aligned sizes. This workaround is enabled by default on all Vulkan drivers to solve an issue in the Vulkan SPEC about the texture-to-texture copies with compressed formats. See #1005 (https://github.com/KhronosGroup/Vulkan-Docs/issues/1005) for more details.
  • vulkan_use_d32s8: https://crbug.com/dawn/286: Vulkan mandates support of either D32_FLOAT_S8 or D24_UNORM_S8. When available the backend will use D32S8 (toggle to on) but setting the toggle to off will make ituse the D24S8 format when possible.
  • disallow_unsafe_apis: http://crbug.com/1138528: Produces validation errors on API entry points or parameter combinations that aren't considered secure yet.
    • [WebGPU Forced Toggles - enabled]
  • disallow_spirv: https://crbug.com/1214923: Disallow usage of SPIR-V completely so that only WGSL is used for shader modules.This is useful to prevent a Chromium renderer process from successfully sendingSPIR-V code to be compiled in the GPU process.
    • [Supported Features]
  • texture-compression-bc
  • pipeline-statistics-query
  • timestamp-query
  • depth-clamping
  • dawn-internal-usages

Version Information

Data exported | 2022-01-26T11:40:58.301Z -- | -- Chrome version | Chrome/97.0.4692.99 Operating system | Windows NT 10.0.22000 Software rendering list URL | https://chromium.googlesource.com/chromium/src/+/d740da257583289dbebd2eb37e8668928fac5ead/gpu/config/software_rendering_list.json Driver bug list URL | https://chromium.googlesource.com/chromium/src/+/d740da257583289dbebd2eb37e8668928fac5ead/gpu/config/gpu_driver_bug_list.json ANGLE commit id | 907d2234409b 2D graphics backend | Skia/97 b4d28b2f35396ae4dd69338254415066629dfd25 Command Line | "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --flag-switches-begin --enable-features=ParallelDownloading,UseDownloadOfflineContentProvider --flag-switches-end

Log Messages

  • GpuProcessHost: The info collection GPU process exited normally. Everything is okay.
mrdoob commented 2 years ago

@superLeaAtWork

@mrdoob hiere the two examples:

image

image

Many many thanks! That really helps to figure out where the issue may be.

Could you also share a screen recording of the changeMaterial example moving the roughness slider from 0 to 1 and back?

felipe-chamas commented 2 years ago

@mrdoob

https://user-images.githubusercontent.com/49691924/151218054-52b1be53-8a43-4f80-95ee-00d25bc1decd.mp4

mrdoob commented 2 years ago

@FelipeR2U Very helpful! Thanks!

mrdoob commented 2 years ago

@superLeaAtWork @FelipeR2U

So @elalish is wondering if this could be a bug with half-float on these GPUs.

I've prepared a couple of links to try:

https://mrdoob-sandbox.glitch.me/pmrem-float.html https://mrdoob-sandbox.glitch.me/pmrem-halffloat.html

Do you mind sharing screenshots of both?

felipe-chamas commented 2 years ago

Sending the image once, but both links on both Chrome and Firefox gave this result:

image

mrdoob commented 2 years ago

Thanks @FelipeR2U 🙏

@elalish anything else you can think of?

superLeaAtWork commented 2 years ago

@mrdoob I have the same result as @FelipeR2U :)

Maybe the information helps you, that it still worked 1-2 weeks ago!

kenrussell commented 2 years ago

@mrdoob @elalish a couple of thoughts:

elalish commented 2 years ago

@kenrussell It's actually a custom 2D texture that is acting as a cubemap (needed custom mipmap sizes). So it sort of looks like a rounding or interpolation error of some kind. The relevent shader chunk is here: https://github.com/mrdoob/three.js/blob/2768965fb58e482e80f78ac95d3d4afcc8ebd895/src/renderers/shaders/ShaderChunk/cube_uv_reflection_fragment.glsl.js

superLeaAtWork commented 2 years ago

Hello,

maybe the information helps you that the wrong appearance only happens when viewing the models in the model-viewer editor.https://modelviewer.dev/editor/

When we integrate the model-viewer into our website and load the same model there, the model appears correct. Also when viewing it on the same PC with the same browser where the model viewer editor is corrupt...

Best regards. Lea

elalish commented 2 years ago

@superLeaAtWork Which version of model-viewer are you integrating with? @FelipeR2U said the break occurred between v1.9 and v1.10. Is that your experience too?

kenrussell commented 2 years ago

Thanks for the information regarding how the cube maps are packed into the texture.

I'm still wondering whether the contents of that custom 2D texture are corrupted, or whether the shader that samples them is producing incorrect answers at what would otherwise be cube map seams.

Is there any way to get answers to these questions in conjunction with someone who can reproduce the problem?

mrdoob commented 2 years ago

@kenrussell

Yes we do:

https://github.com/mrdoob/three.js/blob/1bb26d7dbfb925b6499a3f4e51fd56f53d95ac9f/src/renderers/webgl/WebGLExtensions.js#L52-L75

kenrussell commented 2 years ago

@FelipeR2U the about:gpu information provided above is missing the "Driver Information" section, which is key. Could you please go to about:gpu, click "Copy report to clipboard", paste it on pastebin.com and post the link here? Or, paste it into a plain .txt file and attach it here. Make sure the .txt file has the correct line breaks and isn't just one huge line. Thanks.

kenrussell commented 2 years ago

@aleino-nv per https://twitter.com/PavelBoytchev/status/1492224108318318593 it seems this behavior is triggered when enabling "max anisotropy" in NVIDIA's control panel settings. Does that override any setting of TEXTURE_MAX_ANISOTROPY_EXT by the application (the browser in this case)?

mrdoob commented 2 years ago

@superLeaAtWork @FelipeR2U Have you changed these settings in your graphics card?

image

If so, try changing it to "Let the 3D application decide".

superLeaAtWork commented 2 years ago

Hello,

changing the NVIDIA settings did not change anything: image

Furthermore, I did not change anything in my graphics card settings.

How do I see which version I integrate into the code? On the following link you can see an example of an integrated web viewer: https://www.pfister.ch/de/produkt/28756/kartell-stuhl-masters On this page the visualization is correct.

Regards, Lea

mrdoob commented 2 years ago

@superLeaAtWork Thanks for testing that 🙏

One more test, can you share a screenshot of how this test looks like in your system? https://mrdoob-sandbox.glitch.me/pmrem-nearest-halffloat.html

superLeaAtWork commented 2 years ago

@mrdoob your welcome!

image

This time I do not see any strange lines.

mrdoob commented 2 years ago

Excellent! Thanks!

elalish commented 2 years ago

I'll call this fixed, since the fix is merged upstream in three.js and we'll get it when we pull the next version.

superLeaAtWork commented 2 years ago

Hello,

excellent. Can you just explain the bug in 2 sentences? I'm just curious! :)

Best regards, LEa

elalish commented 2 years ago

Well, the short answer is it's a driver bug we don't understand (and even if we did, it would take a long time to get fixed). It has something to do with anisotropy, float textures, and linear filtering. @mrdoob got around it by setting the minFilter on this texture to Nearest from Linear (which doesn't appear to change the rendered result for working devices). This kind of ridiculous GPU driver workaround is exactly what makes three.js such a valuable project!

mrdoob commented 2 years ago

We recently simplified the internal code that produces the image based lighting and we got a few reports from people with nvidia cards seeing these lines. Turns out the reason these lines appear is because the nvidia settings panel allow the user to override a setting in the code that produces the lighting.