Render video cannot show

[HungNgoCT]

Hi @SuLvXiangXin ,

Thank you for great work.

When I run render script after trained, I can get images in path_render_step. One of color image as below

However, rendering videos cannot show. I use Windows Subsystem for Linux (WSL) for training and rendering.

Reproduce: Train: SCENE=bicycle DATA_DIR=data/bicycle EXP_NAME=bicycle

python train.py --gin_configs=configs/360.gin --gin_bindings="Config.data_dir = '${DATA_DIR}'" --gin_bindings="Config.exp_name = '${EXP_NAME}'" --gin_bindings="Config.factor = 4" --gin_bindings="Config.batch_size = 8192" --gin_bindings="Config.render_chunk_size = 8192"

Render: bash scripts/render_360.sh

File render_360.sh: SCENE=bicycle EXPERIMENT="$SCENE" DATA_ROOT=data DATA_DIR="$DATA_ROOT"/"$SCENE"

accelerate launch render.py \ --gin_configs=configs/360.gin \ --gin_bindings="Config.data_dir = '${DATA_DIR}'" \ --gin_bindings="Config.exp_name = '${EXPERIMENT}'" \ --gin_bindings="Config.render_path = True" \ --gin_bindings="Config.render_path_frames = 120" \ --gin_bindings="Config.render_video_fps = 60" \ --gin_bindings="Config.factor = 4"

Appreciate if you can give me some advice. Many thanks.

[SuLvXiangXin]

@HungNgoCT Is there any *.mp4 generated in your experiement directory? Or you mean you can't open the mp4? I can open the generated mp4 in my macbook, you may try to open it with another video player.

[HungNgoCT]

Thank you @SuLvXiangXin .

There are 4 files *.mp4 generated as the image below. The images in path_renders_step_25000 can be shown. But videos cannot.

The color video is here https://drive.google.com/file/d/1ypW0T10e9ZlZv-N3CCaUQrDDdlZYV3UD/view?usp=sharing

I have tried with Media player, Windows media player... which I used to show *.mp4 files generated from other nerfies.

"you may try to open it with another video player": Yes, if you can play your videos, I will try with other software.

[SuLvXiangXin]

@HungNgoCT It's strange that your video is so large, while my rendered view (factor 8) is only 2MB. I can't open your video either, and I will check the code.

[SuLvXiangXin]

@HungNgoCT Could you try to use the following code to generate your rgb.mp4 and see whether it can show? I can't find any error in the code.

import os
import imageio
from PIL import Image
import numpy as np

# the directory of your rendered images
render_dir = 'exp/bicycle/render/path_renders_step_25000'

# output mp4 path
video_file = 'rgb.mp4'

num_frames = 120
zpad = max(3, len(str(num_frames - 1)))
idx_to_str = lambda idx: str(idx).zfill(zpad)

file_ext = 'png'
print(f'Making video {video_file}...')

writer = imageio.get_writer(video_file, fps=60)
for idx in range(num_frames):
    img_file = os.path.join(render_dir, f'color_{idx_to_str(idx)}.{file_ext}')
    if not os.path.exists(img_file):
        ValueError(f'Image file {img_file} does not exist.')

    img = np.array(Image.open(img_file), dtype=np.float32)
    img = img / 255.
    frame = (np.clip(np.nan_to_num(img), 0., 1.) * 255.).astype(np.uint8)
    writer.append_data(frame)
writer.close()

[HungNgoCT]

@HungNgoCT Could you try to use the following code to generate your rgb.mp4 and see whether it can show? I can't find any error in the code.

import os
import imageio
from PIL import Image
import numpy as np

# the directory of your rendered images
render_dir = 'exp/bicycle/render/path_renders_step_25000'

# output mp4 path
video_file = 'rgb.mp4'

num_frames = 120
zpad = max(3, len(str(num_frames - 1)))
idx_to_str = lambda idx: str(idx).zfill(zpad)

file_ext = 'png'
print(f'Making video {video_file}...')

writer = imageio.get_writer(video_file, fps=60)
for idx in range(num_frames):
    img_file = os.path.join(render_dir, f'color_{idx_to_str(idx)}.{file_ext}')
    if not os.path.exists(img_file):
        ValueError(f'Image file {img_file} does not exist.')

    img = np.array(Image.open(img_file), dtype=np.float32)
    img = img / 255.
    frame = (np.clip(np.nan_to_num(img), 0., 1.) * 255.).astype(np.uint8)
    writer.append_data(frame)
writer.close()

Hi @SuLvXiangXin ,

1. Cannot not show video created from above script using WSL environment. File size is 357 MB (strange as your indication)

2. Can show (file size ~ 2MB) if I run that script in a pure MS Windows environment, as below

https://user-images.githubusercontent.com/110378166/235851820-fb4b3bea-b555-4596-ab2d-931388fc5bcc.mp4

I think problem here is with Windows Subsystem for Linux environment. I am also finding how to solve and update you if I can do it.

[zaf05]

I also encountered this problem, the video cannot be played, I am under Ubuntu, and it is the same with other players

[ZhangWeiSYSU]

@HungNgoCT @zaf05 Hi，I encountered the same problem, and resolved after run the following instruction: pip install imageio[ffmpeg] imageio[pyav] Hope it is helpful for you.

[ZhangWeiSYSU]

您好，您的来信已收到，我会尽快给您回复。