GANWANSHUI / GaussianOcc

GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting
https://ganwanshui.github.io/GaussianOcc/
Apache License 2.0
205 stars 9 forks source link

Render image color from rasterizer when changing use_semantic and semantic #18

Closed RyanPham19092002 closed 1 month ago

RyanPham19092002 commented 2 months ago

Hi, Thanks for your great work I want to ask that,

  1. If I change file nusc-sem-gs.txt at use_semantic = True -> False: After that, I get rgb_marched from occupancy_decoder.py (rgb_marched is rendered_image color was rasterized from diff-gausian-rasterizer) and visualize , but it's tensor full 0 (black images).
  2. If I remain use_semantic = True at file nusc-sem-gs.txt and change file lib/gaussian_renderer/init.py at raster_settings = diff_3d_gaussian_rasterization_semantic.GaussianRasterizationSettings(...) to raster_settings = diff_3d_gaussian_rasterization.GaussianRasterizationSettings(...) And rasterizer = diff_3d_gaussian_rasterization_semantic.GaussianRasterizer(raster_settings=raster_settings) to rasterizer = diff_3d_gaussian_rasterization.GaussianRasterizer(raster_settings=raster_settings) I get the rgb_marched (or colors from rasterizer) like the image below (RGB - depth - color rasterizer, respectively) image

3.If I not change anything, and run script code , rgb_marched has 18 channels (respectively 18 semantic classes) and I don't know how to reduce 18 channels to 3 channels to save the color image as input image.

So I want to ask how can I render RGB images from your rasterizer to visualize instead of load color of input from val_loader at run_vis.py?

Thank you

GANWANSHUI commented 1 month ago

The original rasterizer could render rgb and depth images, you may have a adaptation on it.