WSL2 Vulkan Support Issues Causing LeRobot Example to Fail

Hezhexi2002 commented 3 months ago

System Info

- `lerobot` version: 0.1.0
- Platform: Linux-5.10.102.1-microsoft-standard-WSL2+-x86_64-with-glibc2.31
- Python version: 3.10.14
- Huggingface_hub version: 0.23.4
- Dataset version: 2.20.0
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.3.1+cu121 (True)
- Cuda version: 12010
- Using GPU in script?: yes

Information

[X] One of the scripts in the examples/ folder of LeRobot
[ ] My own task or dataset (give details below)

Reproduction

Steps to reproduce the behavior:

Install and configure Vulkan drivers in WSL2.

sudo apt update
sudo apt install -y vulkan-utils mesa-vulkan-drivers

Set the DISPLAY environment variable for X11 forwarding.

export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):0.0

Verify Vulkan installation:
```
vulkaninfo
```

Try running the LeRobot example script:

python lerobot/scripts/visualize_dataset.py --repo-id lerobot/pusht --episode-index 0

Observe the terminal output for errors.

Error messages:

[2024-06-16T06:48:06Z WARN wgpu_core::instance] DownlevelCapabilities { flags: DownlevelFlags( COMPUTE_SHADERS | FRAGMENT_WRITABLE_STORAGE | INDIRECT_EXECUTION | BASE_VERTEX | READ_ONLY_DEPTH_STENCIL | NON_POWER_OF_TWO_MIPMAPPED_TEXTURES | CUBE_ARRAY_TEXTURES | COMPARISON_SAMPLERS | INDEPENDENT_BLEND | VERTEX_STORAGE | FRAGMENT_STORAGE | MULTISAMPLED_SHADING | DEPTH_TEXTURE_AND_BUFFER_COPIES | WEBGPU_TEXTURE_FORMAT_SUPPORT | BUFFER_BINDINGS_NOT_16_BYTE_ALIGNED | UNRESTRICTED_INDEX_BUFFER | FULL_DRAW_INDEX_UINT32 | DEPTH_BIAS_CLAMP | VIEW_FORMATS | UNRESTRICTED_EXTERNAL_TEXTURE_COPIES | NONBLOCKING_QUERY_RESOLVE | VERTEX_AND_INSTANCE_INDEX_RESPECTS_RESPECTIVE_FIRST_VALUE_IN_INDIRECT_DRAW, ), limits: DownlevelLimits, shader_model: Sm5, } WARNING: lavapipe is not a conformant vulkan implementation, testing use only. [2024-06-16T06:48:07Z WARN egui_winit::clipboard] Failed to initialize arboard clipboard: Unknown error while interacting with the clipboard: X11 server connection timed out because it was unreachable [2024-06-16T06:48:08Z WARN re_renderer::context] Bad software rasterizer detected - expect poor performance and crashes. See: https://www.rerun.io/docs/getting-started/troubleshooting#graphics-issues [2024-06-16T06:48:08Z INFO re_renderer::context] wgpu adapter backend: Vulkan, device_type: Cpu, name: "llvmpipe (LLVM 12.0.0, 256 bits)", driver: "llvmpipe", driver_info: "Mesa 21.2.6 (LLVM 12.0.0)" [2024-06-16T06:48:08Z ERROR re_viewer] Failed to create render context: The given device doesn't support the required downlevel capabilities for the given hardware caps DeviceCaps { tier: FullWebGpuSupport, max_texture_dimension2d: 16384, max_buffer_size: 2147483647, backend_type: WgpuCore }. Required: DownlevelCapabilities { flags: DownlevelFlags(COMPUTE_SHADERS | FRAGMENT_WRITABLE_STORAGE | INDIRECT_EXECUTION | BASE_VERTEX | READ_ONLY_DEPTH_STENCIL | NON_POWER_OF_TWO_MIPMAPPED_TEXTURES | CUBE_ARRAY_TEXTURES | COMPARISON_SAMPLERS | INDEPENDENT_BLEND | VERTEX_STORAGE | ANISOTROPIC_FILTERING | FRAGMENT_STORAGE | MULTISAMPLED_SHADING | DEPTH_TEXTURE_AND_BUFFER_COPIES | WEBGPU_TEXTURE_FORMAT_SUPPORT | BUFFER_BINDINGS_NOT_16_BYTE_ALIGNED | UNRESTRICTED_INDEX_BUFFER | FULL_DRAW_INDEX_UINT32 | DEPTH_BIAS_CLAMP | VIEW_FORMATS | UNRESTRICTED_EXTERNAL_TEXTURE_COPIES | SURFACE_VIEW_FORMATS | NONBLOCKING_QUERY_RESOLVE | VERTEX_AND_INSTANCE_INDEX_RESPECTS_RESPECTIVE_FIRST_VALUE_IN_INDIRECT_DRAW), limits: DownlevelLimits, shader_model: Sm4 } Actual: DownlevelCapabilities { flags: DownlevelFlags(COMPUTE_SHADERS | FRAGMENT_WRITABLE_STORAGE | INDIRECT_EXECUTION | BASE_VERTEX | READ_ONLY_DEPTH_STENCIL | NON_POWER_OF_TWO_MIPMAPPED_TEXTURES | CUBE_ARRAY_TEXTURES | COMPARISON_SAMPLERS | INDEPENDENT_BLEND | VERTEX_STORAGE | FRAGMENT_STORAGE | MULTISAMPLED_SHADING | DEPTH_TEXTURE_AND_BUFFER_COPIES | WEBGPU_TEXTURE_FORMAT_SUPPORT | BUFFER_BINDINGS_NOT_16_BYTE_ALIGNED | UNRESTRICTED_INDEX_BUFFER | FULL_DRAW_INDEX_UINT32 | DEPTH_BIAS_CLAMP | VIEW_FORMATS | UNRESTRICTED_EXTERNAL_TEXTURE_COPIES | NONBLOCKING_QUERY_RESOLVE | VERTEX_AND_INSTANCE_INDEX_RESPECTS_RESPECTIVE_FIRST_VALUE_IN_INDIRECT_DRAW), limits: DownlevelLimits, shader_model: Sm5 }

Expected behavior

I expect to be able to successfully run the LeRobot example script in the WSL2 environment and see the visualization of the data.

aliberts commented 3 months ago

Hi, thank you for reporting this. I'm not familiar with WSL2 driver supports as we don't yet test our library on this system, although it looks like a problem with rerun. This issue might be of interest to you: https://github.com/rerun-io/rerun/issues/1285

We are working on a simpler html dataset viewer right now: #188

Also, are you able to use our docker image?

docker run -ti --gpus all --shm-size "16gb" huggingface/lerobot-gpu:latest

You can use the visualize_dataset.py script as remote from inside the container.

Hezhexi2002 commented 3 months ago

rerun-io/rerun#1285

Thank you for your response and the helpful information, aliberts. I appreciate the suggestion to look into the https://github.com/rerun-io/rerun/issues/1285. I'll investigate this further to see if it provides any insights into resolving the Vulkan support problems I'm experiencing in WSL2. Additionally, I'll try running the example using the Docker image you've provided.

aliberts commented 2 months ago

Closing this for inactivity, feel free to reopen if you still have issues. If you solved it don't hesitate to post your fix here as well to help other users in your situation ;)

huggingface / lerobot