search

xverse-engine / XV3DGS-UEPlugin

A Unreal Engine 5 (UE5) based plugin aiming to provide real-time visulization, management, editing, and scalable hybrid rendering of Guassian Splatting model.
Apache License 2.0
561 stars 73 forks source link
computer-graphics computer-vision radiance-field unreal-engine-5

readme

XVERSE 3D-GS UE Plugin

License

English | 中文

[

](http://xverse.cn/)

Introduction

XVERSE 3D Gaussian Splatting UE plugin（XVERSE 3D-GS UE Plugin）is an Unreal Engine 5 (UE5) based plugin developed by XVERSE Technology Inc. (Zhenshen, China), aiming to provide real-time visualization, management, editing, and scalable hybrid rendering of Gaussian Splatting model, which is a recently arised technique for reconstructing 3D scenes from multiple photos, more details can refer to here.

Currently, our plugin is implemented based on UE5 Niagara, fully exploiting the advantage of the features provided by UE5 to realize efficient rendering and managing for Guassian Splatting models.

Since we are at early access, current accessible features are summarized below:

We will actively release new features in this repo, please stay tuned. Some future updates will contain:

Getting Started

Video Tutorial

Youtube（English）：

B站（中文）：

System Requirements

Plugin download

  1. Open github Release page

  2. Download the zip corresponding to the UE version you are using

  3. Unzip and move folder XV3dGS to folder Plugins

    Demo Data Download Links:

We have provided two sets of test data available for download at:

Included in these downloads are demo_fox_1080p.mp4 and demo_office_1080p.mp4, serving as two sample MP4 files for testing XV3DTools' local training capabilities.

demo_fox_gs.ply and demo_office_gs.ply are 3DGS models trained by XV3DTools, which can be imported into Unreal Engine (UE) using this plugin for real-time rendering.

Open demo project

This repo contains a demo project with an example scene and level

  1. download using git clone

    git clone https://github.com/xverse-engine/XV3DGS-UEPlugin.git

  2. Open XV3DGS.uproject to start UE

  3. Open demo level

Import your guassian splatting model

After v1.1.3, this plugin supports dragging multiple ply files directly into the Content Browser and importing them.

Click on the 'Load Assets' button on the navigation bar or navigate to Window/XV3dgs.

This action will bring up an integrated tool page by default set to load the PLY page. Click on the 'Load PLY' button.

This will trigger a resource selection window to appear.

Choose your Gaussian model file (in .ply format), then it will automaticly generate LOD and loaded into your Content Browser as a blueprint.

Drag the blueprint into Level Viewport and you can see your Gaussian scene.

Local Training on Windows Platform

We develop the XV3DTools to train MP4 videos into Gaussian Splatting ply. XV3DGS-UEPlugin Version XV3DTools Version
1.0.0 ~ 1.1.4 1.1.1
1.1.5 1.1.2

Please Note: Installation of CUDA 11 or higher is required. CUDA Download Link

XV3DTools Usage Steps:

  1. Download XV3DTools tool from the cloud drive to your local machine:
XV3DTools Version Download Link Preview
1.1.1 Baidu Pan, Google Drive
1.1.2 Baidu Pan, Google Drive
  1. Click on the icon to open the interface and switch to the 'training' page.

  1. Click button to set the parameter "XV3DTools Folder" to the local path where XV3DTools tool is located. The path can only be composed of English words. no chinese words and no special words such as "# $ %".

  2. Click button to select the MP4 file for processing and set it as the parameter "Input MP4 Path" (requires video frame rate > 20). The path can only be composed of English words. no chinese words and no special words such as "# $ %".

  3. Click button to choose the output folder and set it as the parameter "Output Folder". The path can only be composed of English words. no chinese words and no special words such as "# $ %".

  4. click Start Training button.

  1. Training takes time. Once completed, locate the trained ply file in the output folder.

Training Speed Reference:

Packaging UE level for Windows Using Our Plugins

You can refer to the following tutorial to package your own UE levels into exe files that can be run on windows System.

packing tutorial

We provide a packaged Windows executable demo, you can download pack_win_exe.zip from release, simply double-click the exe file to start it.

Feature Introduction

RTS (Rotation, Translation, Scaling) Transform

Select the 3D-GS asset and drag it into the scene. then you can Transform it.

You can also select a 3D-GS actor in the level viewport and transform it using GIZMO.

Model Clipping by Gaussian model

  1. Click to select your Gaussian model blueprint, then click to select the "Clip Volume Component", and open the "Visible" box in the "Clip Volume Component".
  1. With the "Clip Volume Component" selected, you can see a crop volume in the window. You can modify the rotation, translation, and scale of the volume by adjusting the Transform attribute. Alternatively, use the keyboard shortcuts WER to visually adjust the GIZMO for rotation, translation, and scale.
  1. Keep the "Clip Volume Component" selected, and choose different cropping options in the Crop Type. The default state is no cropping, but you can change it to crop the inside or outside of the volume. The example below shows cropping the outside of the volume.
  1. Click the "Crop GSPoints" button to clip the specified portion and obtain a clean Niagara.
  1. This clean Niagara can be used for VFX. For more details, please refer to our VFX tutorial (Coming soon).

Dynamic illumination

Dynamic lighting is not default opened.

you can enable it by turning on the withRelighting button.

Supports all the UE light types

The light source can be moved at will

Change color

Modify the AlbedoTint parameter to change the overall color.

LOD

For imported point clouds, LOD is automatically calculated, with the default being to explicitly display the maximum LOD within the limit of 2 million points for Niagara. The parameter "VisWhichLOD" can be used to force the use of a specific LOD.

Dynamic switching of LOD based on the view camera will be supported in future updates.

Release Note

v1.1.5

v1.1.4

v1.1.3

v1.1.2

v1.1.1

v1.1.0

v1.0.1

v1.0.0

Contributors