Error when call ns-render-mesh

[XinyueZ]

Traceback (most recent call last):
  File "/home/user/.local/bin/ns-render-mesh", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/user/sdfstudio/scripts/render_mesh.py", line 257, in entrypoint
    tyro.cli(RenderTrajectory).main()
  File "/home/user/sdfstudio/scripts/render_mesh.py", line 242, in main
    _render_trajectory_video(
  File "/home/user/sdfstudio/scripts/render_mesh.py", line 108, in _render_trajectory_video
    vis.get_render_option().load_from_json("scripts/render.json")
AttributeError: 'NoneType' object has no attribute 'load_from_json'

[niujinshuchong]

Hi, this looks like the problem of open3d. Do you use offscreen rendering?

[flow-specter]

Hi, this looks like the problem of open3d. Do you use offscreen rendering?

hi, can I ask that how to use offscreen rendering?

[rockywind]

Hi, I met the same error.

  File "/rockywin.wang/NeRF/sdfstudio/scripts/render_mesh.py", line 108, in _render_trajectory_video
    vis.get_render_option().load_from_json("scripts/render.json")
AttributeError: 'NoneType' object has no attribute 'load_from_json'

How can we modified the code to fit the headless mode?

[niujinshuchong]

@flow-specter @rockywind You can install xvfb with sudo apt install xvfb and then run xvfb-run -s "-screen 0 640x480x24" ns-render-mesh xxx.

[rockywind]

Hi, thank you for you help! It works for me, but the saved images are all the same. How can I modify the camera trajectory? The below is my script.

xvfb-run -s "-screen 0 640x480x24" ns-render-mesh --meshfile meshes/bakedangelo_barn_colmap_highres.ply --traj interpolate  \
--output-path renders/bakedangelo_barn_colmap_highres.mp4 \
nerfstudio-data --data data/sdfstudio-tanks-and-temples/Barn_colmap

[niujinshuchong]

@rockywind The camera parameters of open3d will be updated every frame: https://github.com/autonomousvision/sdfstudio/blob/master/scripts/render_mesh.py#L154-L171. Could you check if it's updated in your case?

[rockywind]

@niujinshuchong Hi， I found the param.extrinsic is updated, but the render results are all the same. This is my render script.

--output-path outputs/renders/bakedangelo_barn_colmap_highres.mp4 \
nerfstudio-data --data data/sdfstudio-tanks-and-temples/Barn_colmap

This is my train script.

--pipeline.model.level-init 8 --trainer.steps-per-eval-image 5000  \
--pipeline.datamanager.train-num-rays-per-batch 2048 \
--pipeline.datamanager.eval-num-rays-per-batch 512 \
--pipeline.model.sdf-field.use-appearance-embedding True \
--pipeline.model.background-color white --pipeline.model.sdf-field.bias 0.1 \
--pipeline.model.sdf-field.inside-outside False --pipeline.model.background-model grid \
--vis tensorboard --experiment-name barn_colmap_highres  \
--trainer.load-dir outputs/barn_colmap_highres/bakedangelo/2023-06-26_184901/sdfstudio_models \
nerfstudio-data --data data/sdfstudio-tanks-and-temples/Barn_colmap --downscale-factor 1 

The camera's extrinsic is updated.

[flow-specter]

Hi, thanks for your help, but it can't work for me. I met this error ----> Segmentation fault (core dumped)

[yuyingyeh]

Save issue here. The renderings are always from the same camera pose. I'm using a Docker container with Ubuntu.

[rockywind]

Hi, if I want to debug the code how can I add the xvfb-run -s "-screen 0 640x480x24" to vscode launch.json.

@flow-specter @rockywind You can install xvfb with sudo apt install xvfb and then run xvfb-run -s "-screen 0 640x480x24" ns-render-mesh xxx.

[rawmarshmellows]

I believe the issue is due to xvfb, I was able to get headless rendering working by using this Dockerfile, you may need to change your TCNN_CUDA_ARCHITECTURE environmental variable to what your GPU is from the table below:

H100	40X0	30X0	A100	20X0	TITAN V / V100	10X0 / TITAN Xp	9X0	K80
90	89	86	80	75	70	61	52	37

# Define base image.
ARG CUDA_VERSION=11.3.1
ARG OS_VERSION=20.04
ARG USER_ID=1000
# Define base image.
FROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu${OS_VERSION}
ARG CUDA_VERSION
ARG OS_VERSION
ARG USER_ID

# Set environment variables.
## Set non-interactive to prevent asking for user inputs blocking image creation.
ENV DEBIAN_FRONTEND=noninteractive
## Set timezone as it is required by some packages.
ENV TZ=Europe/Berlin
## CUDA architectures, required by tiny-cuda-nn.
ENV TCNN_CUDA_ARCHITECTURES=75
## CUDA Home, required to find CUDA in some packages.
ENV CUDA_HOME="/usr/local/cuda"

# Install required apt packages.
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    build-essential \
    cmake \
    ffmpeg \
    git \
    libatlas-base-dev \
    libboost-filesystem-dev \
    libboost-graph-dev \
    libboost-program-options-dev \
    libboost-system-dev \
    libboost-test-dev \
    libcgal-dev \
    libeigen3-dev \
    libfreeimage-dev \
    libgflags-dev \
    libglew-dev \
    libgoogle-glog-dev \
    libmetis-dev \
    libprotobuf-dev \
    libqt5opengl5-dev \
    libsuitesparse-dev \
    nano \
    protobuf-compiler \
    python3.8-dev \
    python3-pip \
    qtbase5-dev \
    wget

# Install GLOG (required by ceres).
RUN git clone --branch v0.6.0 https://github.com/google/glog.git --single-branch && \
    cd glog && \
    mkdir build && \
    cd build && \
    cmake .. && \
    make -j && \
    make install && \
    cd ../.. && \
    rm -r glog
# Add glog path to LD_LIBRARY_PATH.
ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/lib"

# Install Ceres-solver (required by colmap).
RUN git clone --branch 2.1.0 https://ceres-solver.googlesource.com/ceres-solver.git --single-branch && \
    cd ceres-solver && \
    git checkout $(git describe --tags) && \
    mkdir build && \
    cd build && \
    cmake .. -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF && \
    make -j && \
    make install && \
    cd ../.. && \
    rm -r ceres-solver

# Install colmap.
RUN git clone --branch 3.7 https://github.com/colmap/colmap.git --single-branch && \
    cd colmap && \
    mkdir build && \
    cd build && \
    cmake .. && \
    make -j && \
    make install && \
    cd ../.. && \
    rm -r colmap

# Create non root user and setup environment.
RUN useradd -m -d /home/user -u 1000 user

# Switch to new uer and workdir.
USER 1000:1000
WORKDIR /home/user

# Add local user binary folder to PATH variable.
ENV PATH="${PATH}:/home/user/.local/bin"
SHELL ["/bin/bash", "-c"]

# Upgrade pip and install packages.
RUN python3.8 -m pip install --upgrade pip setuptools pathtools promise
# Install pytorch and submodules.
RUN python3.8 -m pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
# Install tynyCUDNN.
RUN python3.8 -m pip install git+https://github.com/NVlabs/tiny-cuda-nn.git#subdirectory=bindings/torch

# Copy nerfstudio folder and give ownership to user.
ADD . /home/user/nerfstudio
USER root
RUN chown -R user:user /home/user/nerfstudio
USER 1000:1000

# Install nerfstudio dependencies.
RUN cd nerfstudio && \
    python3.8 -m pip install -e . && \
    cd ..

# Change working directory
WORKDIR /workspace

# Code to rebuild open3d for headless rendering
USER root 

RUN apt-get install libosmesa6-dev -y

USER 1000:1000

WORKDIR /home/user

RUN python3.8 -m pip install --upgrade cmake && \
git clone https://github.com/isl-org/Open3D.git && \
cd Open3D/ && mkdir build && cd build && \
/home/user/.local/lib/python3.8/site-packages/cmake/data/bin/cmake -DENABLE_HEADLESS_RENDERING=ON \
                 -DBUILD_GUI=OFF \
                 -DBUILD_WEBRTC=OFF \
                 -DUSE_SYSTEM_GLEW=OFF \
                 -DUSE_SYSTEM_GLFW=OFF \
         .. && \
make -j$(nproc) && \
make install-pip-package

CMD ns-install-cli --mode install && /bin/bash

I had to manually rebuild open3d to be able to do headless rendering the instructions are here

Then while in the sdfstudio directory I could build my image like so: docker build . -t sdfstudio

To run it: docker run -it --runtime nvidia --gpus 'all,"capabilities=compute,utility,graphics"' --entrypoint /bin/bash

After that, I was able to get the headless rendering working with the video.

Currently, I am still in the process of trying to figure out how to use the GPU for the rendering as right now I'm only using the CPU. Any help would be appreciated!