render .py not exporting the images for "render_rotate.mp4"

[ppponpon]

Hi, thank you for sharing your great work! I'm trying it work the repository's code of the "neuralvolumes" and "mvp" of your great works with each experiments data you provided. "neuralvolumes" worked well! Thank you!! I was able to exec train.py and render.py of the "NeuralVolume" code of your previous work,and could get the images sequence of "prog_XXXXXX.jpg" and "render_rotate.mp4" movie file. Then I'm trying it work "mvp" at the same emvironment,and I did success to work train.py and could get the images sequence of "prog_XXXXXX.jpg".And "log.txt","optimparams.pt","aeparams.pt" file too. But I'm trying exec render.py,the image sequence files of consisting for "render_rotate.mp4" are not exported at /tmp/xxxxxxxxxx/ directory.

The error message is [image2 @ 0x56400177b780] Could find no file with path '/tmp/5613023327/%06d.png' and index in the range 0-4 /tmp/5613023327/%06d.png: No such file or directory

Do you have any information about this error?

My environment is ubuntu20.04LTS, Python 3.8.13, GCC 9.4.0, PyTorch 1.10.1+cu113,GPU_A6000. The setup.py has been fixed about cuda arch at the mvpraymarch and the utils directory.

[ppponpon]

console log is

$ python render.py experiments/dryice1/experiment2/config.py render.py experiments/dryice1/experiment2/config.py Output path: experiments/dryice1/experiment2 encoder: Encoder( (pad): ZeroPad2d(padding=(25, 25, 0, 0), value=0.0) (down1): ModuleList( (0): Sequential( (0): Conv2dELR( inch=3, outch=64, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (1): Conv2dELR( inch=64, outch=64, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (2): Conv2dELR( inch=64, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (3): Conv2dELR( inch=128, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (4): Conv2dELR( inch=128, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (5): Conv2dELR( inch=256, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (6): Conv2dELR( inch=256, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=demod, ub=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) ) ) (down2): LinearELR( inch=3072, outch=512, norm=demod, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (mu): LinearELR(inch=512, outch=256, norm=None, act=None) (logstd): LinearELR(inch=512, outch=256, norm=None, act=None) ) decoder: Decoder( (enc): LinearELR(inch=256, outch=256, norm=None, act=None) (rgbdec): ContentDecoder( (mod): Sequential( (0): LinearELR( inch=259, outch=16384, norm=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (1): Reshape() (2): ConvTranspose3dELR( inch=256, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(8, 8, 8), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (3): ConvTranspose3dELR( inch=256, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(16, 16, 16), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (4): ConvTranspose3dELR( inch=128, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(32, 32, 32), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (5): ConvTranspose3dELR( inch=128, outch=64, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(64, 64, 64), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (6): ConvTranspose3dELR(inch=64, outch=3, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(128, 128, 128), act=None) ) ) (alphadec): ContentDecoder( (mod): Sequential( (0): LinearELR( inch=256, outch=16384, norm=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (1): Reshape() (2): ConvTranspose3dELR( inch=256, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(8, 8, 8), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (3): ConvTranspose3dELR( inch=256, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(16, 16, 16), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (4): ConvTranspose3dELR( inch=128, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(32, 32, 32), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (5): ConvTranspose3dELR( inch=128, outch=64, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(64, 64, 64), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (6): ConvTranspose3dELR(inch=64, outch=1, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(128, 128, 128), act=None) ) ) (warpdec): ContentDecoder( (mod): Sequential( (0): LinearELR( inch=256, outch=16384, norm=None, act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (1): Reshape() (2): ConvTranspose3dELR( inch=256, outch=256, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(8, 8, 8), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (3): ConvTranspose3dELR( inch=256, outch=128, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(16, 16, 16), act=LeakyReLU(negative_slope=0.2) (act): LeakyReLU(negative_slope=0.2) ) (4): ConvTranspose3dELR(inch=128, outch=3, kernel_size=4, stride=2, padding=1, wsize=0, norm=None, ub=(32, 32, 32), act=None) ) ) ) raymarcher: Raymarcher() bgmodel: BGModel( (lap): LapImage( (pyr): ModuleList( (0): ImageMod() (1): ImageMod() (2): ImageMod() ) (pyr0): ImageMod() ) ) encoder params: 4920448 decoder params: 87204096 colorcal params: 522 bgmodel params: 14549184 total params: 106674250 encoder.down1.0.0.weight 1.2288e-05 encoder.down1.0.0.bias 2.56e-07 encoder.down1.0.1.weight 0.000262144 encoder.down1.0.1.bias 2.56e-07 encoder.down1.0.2.weight 0.000524288 encoder.down1.0.2.bias 5.12e-07 encoder.down1.0.3.weight 0.001048576 encoder.down1.0.3.bias 5.12e-07 encoder.down1.0.4.weight 0.002097152 encoder.down1.0.4.bias 1.024e-06 encoder.down1.0.5.weight 0.004194304 encoder.down1.0.5.bias 1.024e-06 encoder.down1.0.6.weight 0.004194304 encoder.down1.0.6.bias 1.024e-06 encoder.down2.weight 0.006291456 encoder.down2.bias 2.048e-06 encoder.mu.weight 0.000524288 encoder.mu.bias 1.024e-06 encoder.logstd.weight 0.000524288 encoder.logstd.bias 1.024e-06 decoder.enc.weight 0.000262144 decoder.enc.bias 1.024e-06 decoder.rgbdec.mod.0.weight 0.016973824 decoder.rgbdec.mod.0.bias 6.5536e-05 decoder.rgbdec.mod.2.weight 0.016777216 decoder.rgbdec.mod.2.bias 0.000524288 decoder.rgbdec.mod.3.weight 0.008388608 decoder.rgbdec.mod.3.bias 0.002097152 decoder.rgbdec.mod.4.weight 0.004194304 decoder.rgbdec.mod.4.bias 0.016777216 decoder.rgbdec.mod.5.weight 0.002097152 decoder.rgbdec.mod.5.bias 0.067108864 decoder.rgbdec.mod.6.weight 4.9152e-05 decoder.rgbdec.mod.6.bias 0.025165824 decoder.alphadec.mod.0.weight 0.016777216 decoder.alphadec.mod.0.bias 6.5536e-05 decoder.alphadec.mod.2.weight 0.016777216 decoder.alphadec.mod.2.bias 0.000524288 decoder.alphadec.mod.3.weight 0.008388608 decoder.alphadec.mod.3.bias 0.002097152 decoder.alphadec.mod.4.weight 0.004194304 decoder.alphadec.mod.4.bias 0.016777216 decoder.alphadec.mod.5.weight 0.002097152 decoder.alphadec.mod.5.bias 0.067108864 decoder.alphadec.mod.6.weight 1.6384e-05 decoder.alphadec.mod.6.bias 0.008388608 decoder.warpdec.mod.0.weight 0.016777216 decoder.warpdec.mod.0.bias 6.5536e-05 decoder.warpdec.mod.2.weight 0.016777216 decoder.warpdec.mod.2.bias 0.000524288 decoder.warpdec.mod.3.weight 0.008388608 decoder.warpdec.mod.3.bias 0.002097152 decoder.warpdec.mod.4.weight 9.8304e-05 decoder.warpdec.mod.4.bias 0.000393216 colorcal.weight 1.044e-06 colorcal.bias 1.044e-06 bgmodel.lap.pyr.0.image 0.011091456 bgmodel.lap.pyr.1.image 0.044365824 bgmodel.lap.pyr0.image 0.002739456 0.42669699999999994 B params loaded params fuse done /home/usrX/_NeRF_Test/_mvp/mvp/.venv/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] 0 / 370 (0.1019 iter/sec) 16 / 370 (1.7636 iter/sec) 32 / 370 (1.7403 iter/sec) 48 / 370 (1.7401 iter/sec) 64 / 370 (1.7442 iter/sec) 80 / 370 (1.7496 iter/sec) 96 / 370 (1.7768 iter/sec) 112 / 370 (1.7867 iter/sec) 128 / 370 (1.7914 iter/sec) 144 / 370 (1.7938 iter/sec) 160 / 370 (1.7889 iter/sec) 176 / 370 (1.7727 iter/sec) 192 / 370 (1.7374 iter/sec) 208 / 370 (1.7131 iter/sec) 224 / 370 (1.6889 iter/sec) 240 / 370 (1.6834 iter/sec) 256 / 370 (1.6821 iter/sec) 272 / 370 (1.6893 iter/sec) 288 / 370 (1.6874 iter/sec) 304 / 370 (1.7095 iter/sec) 320 / 370 (1.7107 iter/sec) 336 / 370 (1.7483 iter/sec) 352 / 370 (1.7815 iter/sec) 368 / 370 (0.7527 iter/sec) ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100 [image2 @ 0x55794af33780] Could find no file with path '/tmp/5217110678/%06d.png' and index in the range 0-4 /tmp/5217110678/%06d.png: No such file or directory

[stephenlombardi]

It appears that ffmpeg is throwing this error. Make sure you have write permission in /tmp and also verify that /tmp/{numbers} exists (you may have to comment out this line https://github.com/facebookresearch/mvp/blob/main/utils/videowriter.py#L284 which normally deletes that directory after the render is complete). You can also try to change this line https://github.com/facebookresearch/mvp/blob/main/utils/videowriter.py#L250 from apply_async to apply because apply_async usually silences any errors that occur in the writeimage function so that might give some useful debugging info.

[ppponpon]

Thanks for replying. I tried your advices. Removing videowriter.py#L284, /tmp/{numbers} is created,and continues to exist. But no file is created in it. And I tried derivative idea, /tmp to ./tmp, permission is loose there, but result is same. However, while rewriting the code, try and try, then new exception came out! Exception is :

Exception ignored in: <function Pool.del at 0x7f2472b390d0> Traceback (most recent call last): File "/home/usrX/.pyenv/versions/3.8.13/lib/python3.8/multiprocessing/pool.py", line 268, in del File "/home/usrX/.pyenv/versions/3.8.13/lib/python3.8/multiprocessing/queues.py", line 362, in put AttributeError: 'NoneType' object has no attribute 'dumps'

So I tried your another advice, change the line videowriter.py#L250 from apply_async to apply. But I didn't know how to use "apply" in your multiprocessing style's code. so I rewrite batch method of videowriter.py to single process method. That is:

def batch(self, iternum, itemnum, **kwargs):
    b = itemnum.size(0)

    if "encoding" in self.keyfilter:
        self.encodings.extend(kwargs["encoding"].data.to("cpu").numpy().tolist())

    if "headpose" in self.keyfilter:
        self.headpose.extend(kwargs["headpose"].data.to("cpu").numpy().reshape((
            kwargs["headpose"].size(0), -1)).tolist())

    if "lightpow" in self.keyfilter:
        self.lightpow.extend(kwargs["lightpow"].data.to("cpu").numpy().tolist())

    for i in range(b):
        writeimage(("./tmp/{}".format(self.randid), itemnum[i], 
        {k: kwargs[k][i].data.to("cpu") if isinstance(kwargs[k][i], torch.Tensor) else kwargs[k][i] for k in self.keyfilter}),
        {"cmap": self.cmap, "cmapscale": self.cmapscale, "colorbar": self.colorbarimg, "colcorrect": self.colcorrect},         
              )            
    self.nitems += b

But I'm getting the similar following exceptions!!

/home/usrX/_NeRF_Test/_mvp/mvp/.venv/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.) return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined] Traceback (most recent call last): File "render.py", line 127, in writer.batch(iternum, itemnum + torch.arange(b), datacuda, **output) File "/home/usrX/_NeRF_Test/_mvp/mvp/utils/videowriter.py", line 284, in batch writeimage(("./tmp/{}".format(self.randid), itemnum[i], TypeError: writeimage() missing 1 required positional argument: 'outputs' Exception ignored in: <function Pool.del at 0x7f91291ee160> Traceback (most recent call last): File "/home/usrX/.pyenv/versions/3.8.13/lib/python3.8/multiprocessing/pool.py", line 268, in del File "/home/usrX/.pyenv/versions/3.8.13/lib/python3.8/multiprocessing/queues.py", line 362, in put AttributeError: 'NoneType' object has no attribute 'dumps'

Because of my immaturity...,Sorry. Please teach me how to write the correct arguments of writeimage method in your code style for single task processing manner. Otherwise, please give me how to change the line videowriter.py#L250 from apply_async to apply of your style. Thank you.

[stephenlombardi]

It looks like

writeimage(("./tmp/{}".format(self.randid), itemnum[i], 
        {k: kwargs[k][i].data.to("cpu") if isinstance(kwargs[k][i], torch.Tensor) else kwargs[k][i] for k in self.keyfilter}),
        {"cmap": self.cmap, "cmapscale": self.cmapscale, "colorbar": self.colorbarimg, "colcorrect": self.colcorrect},         
              )

is passing a tuple with 3 elements to the first argument but if you look at the method's signature

def writeimage(randid, itemnum, outputs, **kwargs):

you should be passing these parameters as separate arguments, i.e.,

writeimage("./tmp/{}".format(self.randid), itemnum[i], 
        {k: kwargs[k][i].data.to("cpu") if isinstance(kwargs[k][i], torch.Tensor) else kwargs[k][i] for k in self.keyfilter},
        {"cmap": self.cmap, "cmapscale": self.cmapscale, "colorbar": self.colorbarimg, "colcorrect": self.colcorrect},         
              )

[auroraxs]

@ppponpon, you should set keyfilter=["irgbrec"] in config.py at L218

[revanj]

@ppponpon, you should set keyfilter=["irgbrec"] in config.py at L218

I did nothing else and this fixed it for me. Thanks!

[ppponpon]

@revanj Thanks for replying!,and sorry for the late closing of the issue.I couldn't solve the problem. Thank you for sharing your great works!