USD Viewer: Rendering incorrectly on Linux w/ OpenUSD 23.11

boberfly commented 4 months ago

Hello I am testing out USD Viewer and noticed it is rendering fairly oddly (using the latest master branch as of 2024-2-24 but this is the first time trying it for me)

Screenshot from 2024-02-24 15-54-21

The other USD examples also are not rendering correctly. Other DiligilentSamples seem to be working correctly like GLTF Viewer. Testing this on 2 GPUs using --adapter 0 and 1.

Ubuntu 23.10, building+running under Rocky 9.3 in a distrobox/podman container
OpenUSD 23.11 built with build_usd.py also Rocky 9.3 (GCC 11.4.1)
Primary GPU is AMD Radeon 6900XT using Mesa/RADV NAVI21 driver 23.1.4
Secondary GPU is Nvidia Geforce RTX 2070 w/ Nvidia proprietary driver 535.146.02 (AMD used for display)
DiligentEngine built from instructions online but with https://github.com/DiligentGraphics/DiligentFX/pull/88 applied (might be the culprit due to my fix?) also under Rocky 9.3 container
VULKAN_SDK set to LunarG's Linux release of Vulkan 1.3.275.0

Apologies as I know Hydrogent is a work in progress and I might just be stepping on toes here, but I thought I'd report on it if it could be something else.

I run these by first cwd'ing into ~/dev/DiligentEngine/DiligentSamples/Samples/USDViewer/assets and running ../../../../build/DiligentSamples/Samples/USDViewer/USDViewer --adapter 0

Kind regards

TheMostDiligent commented 4 months ago

Hello! We definitely see way more issues with AMD GPUs than with NVidia. Also, Hydrogent is not very well tested on Linux at this point. Our primary platform is Windows.

Few things:

Can you try to run in Debug mode? Diligent performs very extensive validation in Debug build. Also, Vulkan validation layers may tell something useful.
Can you try running in OpenGL mode (--mode gl command line option)

boberfly commented 4 months ago

Hi @TheMostDiligent

Yep understood, I find the Linux drivers to be really decent these days but I have had other problems like the HdStorm delegate having some OpenGL state issues and having far better luck with the Nvidia card for that (maybe the Mesa GL is more strict while Nvidia's can get away with a lot more leaky states).

For the debug build I am getting a macro error for VERIFY in debug builds, something like:

../DiligentFX/Hydrogent/src/HnRenderDelegate.cpp:174:95: error: macro "VERIFY" requires 3 arguments, but only 1 given
  174 |         VERIFY(RenderDelegateCI.TextureBindingMode == HN_MATERIAL_TEXTURES_BINDING_MODE_LEGACY);

../DiligentFX/Hydrogent/interface/../../../DiligentCore/Common/interface/../../Platforms/Basic/interface/DebugUtilities.hpp:45: note: macro "VERIFY" defined here
   45 | #    define VERIFY(Expr, Message, ...)

I'll investigate further if it's not obvious, maybe another header is using that macro definition.

But I can try the OpenGL mode with the previous release build I've got, and it is the same rendering result above on both the AMD and Nvidia cards for me.

Edit: I changed it from VERIFY to VERIFY_EXPR to fix that one.

boberfly commented 4 months ago

Screenshot from 2024-02-24 17-21-25

Success, at least in debug mode! Rendered with the AMD card here, but Nvidia and vk/gl are also correct now as well.

Some debug from Nvidia VK:

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia ../../../../build-debug/DiligentSamples/Samples/USDViewer/USDViewer --mode vk --adapter 1
Diligent Engine: Info: USD Viewer Arguments:
    USD Path:        
    Use vertex pool: Yes
    Use index pool:  Yes
    Tex atlas dim:   2048
Diligent Engine: Info: Using adapter 1: 'NVIDIA GeForce RTX 2070'
Diligent Engine: Info: User-defined allocator is not provided. Using default allocator.
Diligent Engine: Info: Available Vulkan instance layers: 
    VK_LAYER_VALVE_steam_fossilize_64 1.3.207
    VK_LAYER_VALVE_steam_fossilize_32 1.3.207
    VK_LAYER_VALVE_steam_overlay_32 1.3.207
    VK_LAYER_VALVE_steam_overlay_64 1.3.207
    VK_LAYER_MESA_device_select 1.3.211
    VK_LAYER_NV_optimus 1.3.242
Diligent Engine: Info: Supported Vulkan instance extensions: 
    VK_KHR_device_group_creation 0.0.1
    VK_KHR_display 0.0.23
    VK_KHR_external_fence_capabilities 0.0.1
    VK_KHR_external_memory_capabilities 0.0.1
    VK_KHR_external_semaphore_capabilities 0.0.1
    VK_KHR_get_display_properties2 0.0.1
    VK_KHR_get_physical_device_properties2 0.0.2
    VK_KHR_get_surface_capabilities2 0.0.1
    VK_KHR_surface 0.0.25
    VK_KHR_surface_protected_capabilities 0.0.1
    VK_KHR_wayland_surface 0.0.6
    VK_KHR_xcb_surface 0.0.6
    VK_KHR_xlib_surface 0.0.6
    VK_EXT_acquire_drm_display 0.0.1
    VK_EXT_acquire_xlib_display 0.0.1
    VK_EXT_debug_report 0.0.10
    VK_EXT_debug_utils 0.0.2
    VK_EXT_direct_mode_display 0.0.1
    VK_EXT_display_surface_counter 0.0.1
    VK_EXT_swapchain_colorspace 0.0.4
    VK_EXT_surface_maintenance1 0.0.1
    VK_KHR_portability_enumeration 0.0.1
    VK_LUNARG_direct_driver_loading 0.0.1
Diligent Engine: Warning: Validation layer VK_LAYER_KHRONOS_validation is not available.
Diligent Engine: Info: Using physical device 'NVIDIA GeForce RTX 2070', API version 1.3.242, Driver version 23.584.128.
Diligent Engine: Info: Extensions supported by device 'NVIDIA GeForce RTX 2070': 
    VK_KHR_16bit_storage 0.0.1                       VK_KHR_8bit_storage 0.0.1                          VK_KHR_acceleration_structure 0.0.13
    VK_KHR_bind_memory2 0.0.1                        VK_KHR_buffer_device_address 0.0.1                 VK_KHR_copy_commands2 0.0.1
    VK_KHR_create_renderpass2 0.0.1                  VK_KHR_dedicated_allocation 0.0.3                  VK_KHR_deferred_host_operations 0.0.4
    VK_KHR_depth_stencil_resolve 0.0.1               VK_KHR_descriptor_update_template 0.0.1            VK_KHR_device_group 0.0.4
    VK_KHR_draw_indirect_count 0.0.1                 VK_KHR_driver_properties 0.0.1                     VK_KHR_dynamic_rendering 0.0.1
    VK_KHR_external_fence 0.0.1                      VK_KHR_external_fence_fd 0.0.1                     VK_KHR_external_memory 0.0.1
    VK_KHR_external_memory_fd 0.0.1                  VK_KHR_external_semaphore 0.0.1                    VK_KHR_external_semaphore_fd 0.0.1
    VK_KHR_format_feature_flags2 0.0.2               VK_KHR_fragment_shader_barycentric 0.0.1           VK_KHR_fragment_shading_rate 0.0.2
    VK_KHR_get_memory_requirements2 0.0.1            VK_KHR_global_priority 0.0.1                       VK_KHR_image_format_list 0.0.1
    VK_KHR_imageless_framebuffer 0.0.1               VK_KHR_maintenance1 0.0.2                          VK_KHR_maintenance2 0.0.1
    VK_KHR_maintenance3 0.0.1                        VK_KHR_maintenance4 0.0.2                          VK_KHR_multiview 0.0.1
    VK_KHR_pipeline_executable_properties 0.0.1      VK_KHR_pipeline_library 0.0.1                      VK_KHR_present_id 0.0.1
    VK_KHR_present_wait 0.0.1                        VK_KHR_push_descriptor 0.0.2                       VK_KHR_ray_query 0.0.1
    VK_KHR_ray_tracing_maintenance1 0.0.1            VK_KHR_ray_tracing_pipeline 0.0.1                  VK_KHR_relaxed_block_layout 0.0.1
    VK_KHR_sampler_mirror_clamp_to_edge 0.0.3        VK_KHR_sampler_ycbcr_conversion 0.0.14             VK_KHR_separate_depth_stencil_layouts 0.0.1
    VK_KHR_shader_atomic_int64 0.0.1                 VK_KHR_shader_clock 0.0.1                          VK_KHR_shader_draw_parameters 0.0.1
    VK_KHR_shader_float16_int8 0.0.1                 VK_KHR_shader_float_controls 0.0.4                 VK_KHR_shader_integer_dot_product 0.0.1
    VK_KHR_shader_non_semantic_info 0.0.1            VK_KHR_shader_subgroup_extended_types 0.0.1        VK_KHR_shader_subgroup_uniform_control_flow 0.0.1
    VK_KHR_shader_terminate_invocation 0.0.1         VK_KHR_shared_presentable_image 0.0.1              VK_KHR_spirv_1_4 0.0.1
    VK_KHR_storage_buffer_storage_class 0.0.1        VK_KHR_swapchain 0.0.70                            VK_KHR_swapchain_mutable_format 0.0.1
    VK_KHR_synchronization2 0.0.1                    VK_KHR_timeline_semaphore 0.0.2                    VK_KHR_uniform_buffer_standard_layout 0.0.1
    VK_KHR_variable_pointers 0.0.1                   VK_KHR_video_decode_h264 0.0.8                     VK_KHR_video_decode_h265 0.0.7
    VK_KHR_video_decode_queue 0.0.7                  VK_KHR_video_queue 0.0.8                           VK_KHR_vulkan_memory_model 0.0.3
    VK_KHR_workgroup_memory_explicit_layout 0.0.1    VK_KHR_zero_initialize_workgroup_memory 0.0.1      VK_EXT_4444_formats 0.0.1
    VK_EXT_attachment_feedback_loop_layout 0.0.2     VK_EXT_blend_operation_advanced 0.0.2              VK_EXT_border_color_swizzle 0.0.1
    VK_EXT_buffer_device_address 0.0.2               VK_EXT_calibrated_timestamps 0.0.2                 VK_EXT_color_write_enable 0.0.1
    VK_EXT_conditional_rendering 0.0.2               VK_EXT_conservative_rasterization 0.0.1            VK_EXT_custom_border_color 0.0.12
    VK_EXT_depth_clamp_zero_one 0.0.1                VK_EXT_depth_clip_control 0.0.1                    VK_EXT_depth_clip_enable 0.0.1
    VK_EXT_depth_range_unrestricted 0.0.1            VK_EXT_descriptor_buffer 0.0.1                     VK_EXT_descriptor_indexing 0.0.2
    VK_EXT_discard_rectangles 0.0.2                  VK_EXT_display_control 0.0.1                       VK_EXT_extended_dynamic_state 0.0.1
    VK_EXT_extended_dynamic_state2 0.0.1             VK_EXT_extended_dynamic_state3 0.0.2               VK_EXT_external_memory_dma_buf 0.0.1
    VK_EXT_external_memory_host 0.0.1                VK_EXT_fragment_shader_interlock 0.0.1             VK_EXT_global_priority 0.0.2
    VK_EXT_global_priority_query 0.0.1               VK_EXT_graphics_pipeline_library 0.0.1             VK_EXT_host_query_reset 0.0.1
    VK_EXT_image_2d_view_of_3d 0.0.1                 VK_EXT_image_drm_format_modifier 0.0.2             VK_EXT_image_robustness 0.0.1
    VK_EXT_image_sliced_view_of_3d 0.0.1             VK_EXT_image_view_min_lod 0.0.1                    VK_EXT_index_type_uint8 0.0.1
    VK_EXT_inline_uniform_block 0.0.1                VK_EXT_line_rasterization 0.0.1                    VK_EXT_load_store_op_none 0.0.1
    VK_EXT_memory_budget 0.0.1                       VK_EXT_memory_priority 0.0.1                       VK_EXT_mesh_shader 0.0.1
    VK_EXT_multi_draw 0.0.1                          VK_EXT_mutable_descriptor_type 0.0.1               VK_EXT_non_seamless_cube_map 0.0.1
    VK_EXT_opacity_micromap 0.0.2                    VK_EXT_pageable_device_local_memory 0.0.1          VK_EXT_pci_bus_info 0.0.2
    VK_EXT_physical_device_drm 0.0.1                 VK_EXT_pipeline_creation_cache_control 0.0.3       VK_EXT_pipeline_creation_feedback 0.0.1
    VK_EXT_pipeline_library_group_handles 0.0.1      VK_EXT_pipeline_robustness 0.0.1                   VK_EXT_post_depth_coverage 0.0.1
    VK_EXT_primitive_topology_list_restart 0.0.1     VK_EXT_primitives_generated_query 0.0.1            VK_EXT_private_data 0.0.1
    VK_EXT_provoking_vertex 0.0.1                    VK_EXT_queue_family_foreign 0.0.1                  VK_EXT_robustness2 0.0.1
    VK_EXT_sample_locations 0.0.1                    VK_EXT_sampler_filter_minmax 0.0.2                 VK_EXT_scalar_block_layout 0.0.1
    VK_EXT_separate_stencil_usage 0.0.1              VK_EXT_shader_atomic_float 0.0.1                   VK_EXT_shader_demote_to_helper_invocation 0.0.1
    VK_EXT_shader_image_atomic_int64 0.0.1           VK_EXT_shader_module_identifier 0.0.1              VK_EXT_shader_object 0.0.1
    VK_EXT_shader_subgroup_ballot 0.0.1              VK_EXT_shader_subgroup_vote 0.0.1                  VK_EXT_shader_viewport_index_layer 0.0.1
    VK_EXT_subgroup_size_control 0.0.2               VK_EXT_texel_buffer_alignment 0.0.1                VK_EXT_tooling_info 0.0.1
    VK_EXT_transform_feedback 0.0.1                  VK_EXT_vertex_attribute_divisor 0.0.3              VK_EXT_vertex_input_dynamic_state 0.0.2
    VK_EXT_ycbcr_2plane_444_formats 0.0.1            VK_EXT_ycbcr_image_arrays 0.0.1                    VK_NV_clip_space_w_scaling 0.0.1
    VK_NV_compute_shader_derivatives 0.0.1           VK_NV_cooperative_matrix 0.0.1                     VK_NV_copy_memory_indirect 0.0.1
    VK_NV_corner_sampled_image 0.0.2                 VK_NV_coverage_reduction_mode 0.0.1                VK_NV_cuda_kernel_launch 0.0.2
    VK_NV_dedicated_allocation 0.0.1                 VK_NV_dedicated_allocation_image_aliasing 0.0.1    VK_NV_device_diagnostic_checkpoints 0.0.2
    VK_NV_device_diagnostics_config 0.0.2            VK_NV_device_generated_commands 0.0.3              VK_NV_fill_rectangle 0.0.1
    VK_NV_fragment_coverage_to_color 0.0.1           VK_NV_fragment_shader_barycentric 0.0.1            VK_NV_fragment_shading_rate_enums 0.0.1
    VK_NV_framebuffer_mixed_samples 0.0.1            VK_NV_geometry_shader_passthrough 0.0.1            VK_NV_inherited_viewport_scissor 0.0.1
    VK_NV_linear_color_attachment 0.0.1              VK_NV_memory_decompression 0.0.1                   VK_NV_mesh_shader 0.0.1
    VK_NV_ray_tracing 0.0.3                          VK_NV_ray_tracing_invocation_reorder 0.0.1         VK_NV_representative_fragment_test 0.0.2
    VK_NV_sample_mask_override_coverage 0.0.1        VK_NV_scissor_exclusive 0.0.2                      VK_NV_shader_image_footprint 0.0.2
    VK_NV_shader_sm_builtins 0.0.1                   VK_NV_shader_subgroup_partitioned 0.0.1            VK_NV_shading_rate_image 0.0.3
    VK_NV_viewport_array2 0.0.1                      VK_NV_viewport_swizzle 0.0.1                       VK_NVX_binary_import 0.0.1
    VK_NVX_image_view_handle 0.0.2                   VK_NVX_multiview_per_view_attributes 0.0.1         VK_AMD_buffer_marker 0.0.1
Diligent Engine: Info: GPU dynamic heap created. Total buffer size: 8.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 16.00 MB
Diligent Engine: Info: Requested color buffer format TEX_FORMAT_RGBA8_UNORM_SRGB is not supported by the surface and will be replaced with TEX_FORMAT_BGRA8_UNORM_SRGB
Diligent Engine: Info: Using SURFACE_TRANSFORM_IDENTITY swap chain pretransform
Diligent Engine: Info: Using VK_PRESENT_MODE_FIFO_KHR swap chain present mode
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 16.00 MB
Diligent Engine: Info: Allocated new descriptor pool
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (32.00 MB, type idx: 1). Current allocated size: 48.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 32.00 MB
Diligent Engine: Info: Render state cache: Added shader 'Full screen triangle VS [2B6D4A006E646756DC9EF98D2951B22E]'.
Diligent Engine: Info: Render state cache: Added shader 'Precompute BRDF PS [FC5A067350FDA2AA9ED032BD55C2A51E]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Precompute BRDF LUT PSO [CE04CCD1666E6A3151E047417B848050]'.
Diligent Engine: Info: Render state cache: Added shader 'Cubemap face VS [A0C0A0677060FDB177C8484A445490A1]'.
Diligent Engine: Info: Render state cache: Added shader 'Precompute irradiance cube map PS [566864DCDC3090124B2C74C91257701F]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Precompute irradiance cube PSO [7848FFC4180F8FAF9A9B6273C3C16449]'.
Diligent Engine: Info: Render state cache: Added shader 'Cubemap face VS [E6B7A516AF2FED22B2E80BC484A02B76]'.
Diligent Engine: Info: Render state cache: Added shader 'Prefilter environment map PS [5E68909AC49820B426D95E81B08FFEB5]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Prefilter environment map PSO [FA4C7CE2817449A09FF9BF29CA72BFCE]'.
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 48.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (32.00 MB, type idx: 1). Current allocated size: 80.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (32.00 MB, type idx: 2). Current allocated size: 80.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (32.00 MB, type idx: 1). Current allocated size: 112.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (32.00 MB, type idx: 2). Current allocated size: 112.00 MB
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/eNCqvCkKqhectVO'
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 128.00 MB
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/zVVmDVDPlfMpRxG'
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 128.00 MB
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/YdIpDxDXbAsVHMQ'
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/LKCtauoVDdOnIYU'
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 144.00 MB
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/zeOYUYoxmqYjjxm'
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/jCwEUxocVEcpgfO'
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (32.00 MB, type idx: 1). Current allocated size: 176.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (32.00 MB, type idx: 2). Current allocated size: 160.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 192.00 MB
Diligent Engine: Info: Skipping unknown texture 'opacity' in material '/tracking_node_placeholder/qvwQqyyuHuuHkUm/SYbdCiSESBmFjUo'
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 176.00 MB
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Scene color texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Mesh ID texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Motion vectors texture
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 208.00 MB
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Normal texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Base color texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Material texture
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 224.00 MB
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 IBL texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Selection depth buffer texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Depth buffer 0 texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Depth buffer 1 texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Motion vectors 1 texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Closest selected location 0 texture
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 240.00 MB
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Closest selected location 1 texture
Diligent Engine: Info: HnBeginFrameTask: created 1024x768 Jittered final color texture
Diligent Engine: Info: Render state cache: Added shader 'Axes VS [D4DD1C68606B887268902212C6649020]'.
Diligent Engine: Info: Render state cache: Added shader 'Axes PS [7B0C0EBDCE24B065FE8611B53A11FCF7]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Axes [99EF6E8A954FB0BFE82421B849A500D5]'.
Diligent Engine: Info: Render state cache: Added shader 'Full-screen Triangle VS [B14FAE0C385F62795164509B764D6E02]'.
Diligent Engine: Info: Render state cache: Added shader 'Init Closest Selected Location PS [5B86FF52C86B8DA77F23FE71F5824D4D]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Init closest selection [E7DB5294403CE3D9BAFBB46B1C0130B1]'.
Diligent Engine: Info: Render state cache: Added shader 'Update Closest Selected Location PS [CD656B2AA1160EECB0D9FAF3B1545852]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Update closest selection [5ADCDF1BEDF8FD4B96D02DD893BE8A4E]'.
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 256.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 272.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 288.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 304.00 MB
Diligent Engine: Info: Render state cache: Added shader 'Full-screen Triangle VS [47FFBB06317D5FE4CB2DCAE71AE26AE3]'.
Diligent Engine: Info: Render state cache: Added shader 'Post-process PS [8D109D7CE5C5423F9D5AFFAFD5ABFA65]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Post process [0F0DED8A402229FECC01B4E0EF002C35]'.
Diligent Engine: Info: Render state cache: Added shader 'Full-screen Triangle VS [8721369C249A455FE378E7D95F498D7F]'.
Diligent Engine: Info: Render state cache: Added shader 'PostCopy Frame PS [5D1F5978DD6CE59EE7F5FF0D4F67C7FD]'.
Diligent Engine: Info: Render state cache: Added pipeline 'Copy Frame [6F41C88DF193DCAA2CC8A69E03DE00DF]'.
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 320.00 MB
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new device-local page. (16.00 MB, type idx: 1). Current allocated size: 336.00 MB
Diligent Engine: Info: Dynamic buffer: expanding dynamic buffer 'Hydrogent index pool' from 4.0 KB to 8.0 MB. Version: 2
Diligent Engine: Info: VulkanMemoryManager 'Global resource memory manager': created new host-visible page. (16.00 MB, type idx: 2). Current allocated size: 192.00 MB
Diligent Engine: Info: Render state cache: Added shader 'PBR VS [BA1DD0B7BF00C0B83C3A1D37A9734349]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR PS [0E501AA1DABB7A3EA398E4743FE89F17]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - backface culling [3414D001593ADBD8D8886196B475805D]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - no culling [B98BC4BFACC2C81B2F50F35EC2B58B9A]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - backface culling [70367173107D4B9DF04FD9855358E3DB]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - no culling [B523B5A8E36999C6CFF4ABF95B722100]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR VS [8C2BA6D84701658B50798A082E7FC52F]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR PS [675265DD97745C2873229B842DA9B1F2]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - backface culling [B84758CEBD7515A0BEA27E1D58285B7B]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - no culling [6C6002D294C8E859C570D40B087FD197]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - backface culling [FA34C8075330DC2F55389290B95D52CC]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - no culling [D07E72AB32A4DD0804EF50AD3F28C0C3]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR VS [2F9794037A8EAF670398C5CC323F95BF]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR PS [9BAD9C86624DE8DA6CF6C9C573BCFA8B]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - backface culling [3641EE733F8DF8CF511C25A211EF25DE]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - no culling [DEE65F0490E13D340E4F3CE7545E862F]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - backface culling [CFF7950C3EB0BECF58320C5B4D81368C]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - no culling [7C9792A2C3FBBCFF554CCD070125885A]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR VS [5F12D27FC8460EF353FD559119797413]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR PS [1B221F67F3C6BA6178540E2351CE6C6F]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - backface culling [7A196A6D533CA55921BE1F33ACFEAFBE]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - no culling [7FC34BB77A1ED3770F5FC91EF2D5C564]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - backface culling [036ADBEA2B4501B353CF2F942EFC808B]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - no culling [2306C412065C1A9C77DB10931B44CF41]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR VS [F0ADB68044F40275A4EF3D288030B376]'.
Diligent Engine: Info: Render state cache: Added shader 'PBR PS [CF4EB1E307EA5EB7D9E29EB2E4E40E2B]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - backface culling [F490E547A37A2C9A147C33B7349DB49E]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - opaque - no culling [17B3DB4AD0DB51960AE3CD38C344838A]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - backface culling [E5CF2569FD0F27DAFEEE3847FD0092DC]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PBR PSO - blend - no culling [1958ED280314D912A53B546A2AEB8F60]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeBlueNoiseTexturePS [3376C9D590DF6F3CD825233ABA22D287]'.
Diligent Engine: Info: Render state cache: Added pipeline 'PreparePostFX::ComputeBlueNoiseTexture [2FDC2F1B4626641C670230C5DCEF6E68]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeHierarchicalDepthBufferPS [0B4FC73EFCFEB4970C30F6A2D3512D02]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeHierarchicalDepthBuffer [B379FE6CBE93F5061AAF11DB5B238488]'.
Diligent Engine: Info: Render state cache: Added shader 'CopyDepthPS [135E4A10BE298902476BCAF26829EB9B]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeStencilMaskAndExtractRoughnessPS [67E17FB53A941AC81C2FC534FC1703FD]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeStencilMaskAndExtractRoughness [AC3192DC127319148C05297D90BB5C53]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeIntersectionPS [10843335E5FFF851E14C1CB040EAEF9C]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeIntersection [8BAFBEB7D014F3C5ED6CB29271A20CE9]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeSpatialReconstructionPS [ECB99CF2DCD2D451C41F774214ACB33A]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeSpatialReconstruction [7DD18132144B026CF741A0DED8446695]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeTemporalAccumulationPS [2B5712311F3ADC6FD021BD57F3A0DAB4]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeTemporalAccumulation [3E65B9C4B980506822022BB3F0777583]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeBilateralCleanupPS [805999C589B47AB91ED66955F8733097]'.
Diligent Engine: Info: Render state cache: Added pipeline 'ScreenSpaceReflection::ComputeBilateralCleanup [7FECCDD2E4306A526CCA759DCF11CC8A]'.
Diligent Engine: Info: Render state cache: Added shader 'ComputeTemporalAccumulationPS [2927A00EF8BEDB48418CB6B52E644443]'.
Diligent Engine: Info: Render state cache: Added pipeline 'TemporalAntiAliasing::ComputeTemporalAccumulation [542ED943A975611EDBFBAEB4B8EF3D3A]'.
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1
Diligent Engine: Info: Using SURFACE_TRANSFORM_IDENTITY swap chain pretransform
Diligent Engine: Info: Using VK_PRESENT_MODE_IMMEDIATE_KHR swap chain present mode
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1
Diligent Engine: ERROR: Active render pass is incompatible with PSO 'TemporalAntiAliasing::ComputeTemporalAccumulation'. This indicates the mismatch between the number and/or format of bound render targets and/or depth stencil buffer and the PSO. Vulkan requires exact match.
    Bound render targets (1): TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: <Not set>; Sample count: 1
    PSO: render targets (1):  TEX_FORMAT_BGRA8_UNORM_SRGB; DSV: TEX_FORMAT_UNKNOWN (read-only); Sample count: 1

TheMostDiligent commented 4 months ago

Yes, looks like there are some issues on Vulkan - I will take a look.

TheMostDiligent commented 4 months ago

Can you try getting the latest version and see if it works better?

boberfly commented 4 months ago

Hi @TheMostDiligent

I've just made some quick tests with the 6900XT, and I can see that the Vulkan errors are now gone in debug mode.

However I think release mode things have gone a little worse: Screenshot from 2024-03-03 14-37-50 OpenGL mode is similar as well.

I'll try the Nvidia soon, unfortunately distrobox seems to 'drop' the nvidia driver each time I reboot and I need to re-create the container to support the nvidia driver each time...

TheMostDiligent commented 4 months ago

Debug still works OK? I can't really immediately think of any difference between Debug and Release that may cause this.

boberfly commented 4 months ago

Just skimming through Renderdoc with debug vs release here.

Screenshot from 2024-03-03 18-57-55 Looks like the textures/samplers aren't in the bind. I did see there's 3 states of bindless, texture array and legacy for USDRenderer - perhaps the logic here might be funny under debug modes vs release? I'm pretty sure I'm on hardware which can do bindless though...

boberfly commented 4 months ago

Hi @TheMostDiligent

I also noticed normals and UVs were not binding so it gave me a hint that the PSO is probably not getting created like the debug one, and so I focused on the HnRenderPass::UpdateDrawListItemGPUResources() function.

https://github.com/DiligentGraphics/DiligentFX/blob/master/Hydrogent/src/HnRenderPass.cpp#L756

I put a bunch of log prints to see if we are getting nullptrs for them and we are not - but the act of putting prints in here fixes things!

Screenshot from 2024-03-03 20-30-33

So a crude fix was I just put a usleep(1); on that line to see if it is just some race condition, and this fixes things also. OpenGL doesn't quite get there though (textures look odd but it's better than before) but it fixes Vulkan completely.

Hopefully the following discovery gives enough hints as to what is happening and where to fix the race condition.

Cheers

TheMostDiligent commented 4 months ago

Thanks for the investigation. HnRenderPass::UpdateDrawListItemGPUResources() is really called from a single thread, so there might be some kind of synchronization issue in the driver. Did you try to run with Vulkan validation layers enabled - is there anything in the output?

DiligentGraphics / DiligentSamples

USD Viewer: Rendering incorrectly on Linux w/ OpenUSD 23.11 #141