search

mmp / pbrt-v4

Source code to pbrt, the ray tracer described in the forthcoming 4th edition of the "Physically Based Rendering: From Theory to Implementation" book.
https://pbrt.org
Apache License 2.0
2.8k stars 429 forks source link

imgtool denoiser do not properly work! #169

Closed jakoaltrd closed 2 years ago

jakoaltrd commented 3 years ago

it is common to use Optix's denoiser to denoise a img with low spps, since mitsuba2 also use optix to render, it should be able to use pbrt's optix denoiser imgtool to denoise the output with low spps, so I made a experiment:

I use mitusba2 render a rgb image, then I use imgtool to denoise it, but the output looks the same as the input, it seems pbrt's optix denoiser can't work with external sources? why is that? output: image input: image

mmp commented 3 years ago

There's nothing in pbrt's use of the optix denoiser that checks where the image comes from.

It's not possible to have useful response to this bug report without:

jakoaltrd commented 3 years ago

_

<integrator type="path">
    <integer name="max_depth" value="6"/>
</integrator>

<sensor type="perspective">
    <string name="fov_axis" value="smaller"/>
    <float name="near_clip" value="10"/>
    <float name="far_clip" value="2800"/>
    <float name="focus_distance" value="1000"/>
    <float name="fov" value="39.3077"/>
    <transform name="to_world">
        <lookat origin="278, 273, -800"
                target="278, 273, -799"
                up    ="  0,   1,    0"/>
    </transform>
    <sampler type="independent">  <!-- ldsampler -->
        <integer name="sample_count" value="64"/>
    </sampler>
    <film type="hdrfilm">
        <integer name="width" value="256"/>
        <integer name="height" value="256"/>
        <rfilter type="gaussian"/>
    </film>
</sensor>
<bsdf type="diffuse" id="box">
    <spectrum name="reflectance" value="400:0.343, 404:0.445, 408:0.551, 412:0.624, 416:0.665, 420:0.687, 424:0.708, 428:0.723, 432:0.715, 436:0.71, 440:0.745, 444:0.758, 448:0.739, 452:0.767, 456:0.777, 460:0.765, 464:0.751, 468:0.745, 472:0.748, 476:0.729, 480:0.745, 484:0.757, 488:0.753, 492:0.75, 496:0.746, 500:0.747, 504:0.735, 508:0.732, 512:0.739, 516:0.734, 520:0.725, 524:0.721, 528:0.733, 532:0.725, 536:0.732, 540:0.743, 544:0.744, 548:0.748, 552:0.728, 556:0.716, 560:0.733, 564:0.726, 568:0.713, 572:0.74, 576:0.754, 580:0.764, 584:0.752, 588:0.736, 592:0.734, 596:0.741, 600:0.74, 604:0.732, 608:0.745, 612:0.755, 616:0.751, 620:0.744, 624:0.731, 628:0.733, 632:0.744, 636:0.731, 640:0.712, 644:0.708, 648:0.729, 652:0.73, 656:0.727, 660:0.707, 664:0.703, 668:0.729, 672:0.75, 676:0.76, 680:0.751, 684:0.739, 688:0.724, 692:0.73, 696:0.74, 700:0.737"/>
</bsdf>
<bsdf type="diffuse" id="white">
    <spectrum name="reflectance" value="400:0.343, 404:0.445, 408:0.551, 412:0.624, 416:0.665, 420:0.687, 424:0.708, 428:0.723, 432:0.715, 436:0.71, 440:0.745, 444:0.758, 448:0.739, 452:0.767, 456:0.777, 460:0.765, 464:0.751, 468:0.745, 472:0.748, 476:0.729, 480:0.745, 484:0.757, 488:0.753, 492:0.75, 496:0.746, 500:0.747, 504:0.735, 508:0.732, 512:0.739, 516:0.734, 520:0.725, 524:0.721, 528:0.733, 532:0.725, 536:0.732, 540:0.743, 544:0.744, 548:0.748, 552:0.728, 556:0.716, 560:0.733, 564:0.726, 568:0.713, 572:0.74, 576:0.754, 580:0.764, 584:0.752, 588:0.736, 592:0.734, 596:0.741, 600:0.74, 604:0.732, 608:0.745, 612:0.755, 616:0.751, 620:0.744, 624:0.731, 628:0.733, 632:0.744, 636:0.731, 640:0.712, 644:0.708, 648:0.729, 652:0.73, 656:0.727, 660:0.707, 664:0.703, 668:0.729, 672:0.75, 676:0.76, 680:0.751, 684:0.739, 688:0.724, 692:0.73, 696:0.74, 700:0.737"/>
</bsdf>
<bsdf type="diffuse" id="red">
    <spectrum name="reflectance" value="400:0.04, 404:0.046, 408:0.048, 412:0.053, 416:0.049, 420:0.05, 424:0.053, 428:0.055, 432:0.057, 436:0.056, 440:0.059, 444:0.057, 448:0.061, 452:0.061, 456:0.06, 460:0.062, 464:0.062, 468:0.062, 472:0.061, 476:0.062, 480:0.06, 484:0.059, 488:0.057, 492:0.058, 496:0.058, 500:0.058, 504:0.056, 508:0.055, 512:0.056, 516:0.059, 520:0.057, 524:0.055, 528:0.059, 532:0.059, 536:0.058, 540:0.059, 544:0.061, 548:0.061, 552:0.063, 556:0.063, 560:0.067, 564:0.068, 568:0.072, 572:0.08, 576:0.09, 580:0.099, 584:0.124, 588:0.154, 592:0.192, 596:0.255, 600:0.287, 604:0.349, 608:0.402, 612:0.443, 616:0.487, 620:0.513, 624:0.558, 628:0.584, 632:0.62, 636:0.606, 640:0.609, 644:0.651, 648:0.612, 652:0.61, 656:0.65, 660:0.638, 664:0.627, 668:0.62, 672:0.63, 676:0.628, 680:0.642, 684:0.639, 688:0.657, 692:0.639, 696:0.635, 700:0.642"/>
</bsdf>
<bsdf type="diffuse" id="green">
    <spectrum name="reflectance" value="400:0.092, 404:0.096, 408:0.098, 412:0.097, 416:0.098, 420:0.095, 424:0.095, 428:0.097, 432:0.095, 436:0.094, 440:0.097, 444:0.098, 448:0.096, 452:0.101, 456:0.103, 460:0.104, 464:0.107, 468:0.109, 472:0.112, 476:0.115, 480:0.125, 484:0.14, 488:0.16, 492:0.187, 496:0.229, 500:0.285, 504:0.343, 508:0.39, 512:0.435, 516:0.464, 520:0.472, 524:0.476, 528:0.481, 532:0.462, 536:0.447, 540:0.441, 544:0.426, 548:0.406, 552:0.373, 556:0.347, 560:0.337, 564:0.314, 568:0.285, 572:0.277, 576:0.266, 580:0.25, 584:0.23, 588:0.207, 592:0.186, 596:0.171, 600:0.16, 604:0.148, 608:0.141, 612:0.136, 616:0.13, 620:0.126, 624:0.123, 628:0.121, 632:0.122, 636:0.119, 640:0.114, 644:0.115, 648:0.117, 652:0.117, 656:0.118, 660:0.12, 664:0.122, 668:0.128, 672:0.132, 676:0.139, 680:0.144, 684:0.146, 688:0.15, 692:0.152, 696:0.157, 700:0.159"/>
</bsdf>
<bsdf type="diffuse" id="light">
    <spectrum name="reflectance" value="400:0.78, 500:0.78, 600:0.78, 700:0.78"/>
</bsdf>
<shape type="obj">
    <string name="filename" value="meshes/cbox_luminaire.obj"/>
    <transform name="to_world">
        <translate x="0" y="-0.5" z="0"/>
    </transform>
    <ref id="light"/>
    <emitter type="area">
        <spectrum name="radiance" value="400:0, 500:8, 600:15.6, 700:18.4"/>
    </emitter>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_floor.obj"/>
    <ref id="white"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_ceiling.obj"/>
    <ref id="white"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_back.obj"/>
    <ref id="white"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_greenwall.obj"/>
    <ref id="green"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_redwall.obj"/>
    <ref id="red"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_smallbox.obj"/>
    <ref id="box"/>
</shape>
<shape type="obj">
    <string name="filename" value="meshes/cbox_largebox.obj"/>
    <ref id="box"/>
</shape>

_ this is the cbox.xml file. what I did is using mitsuba2 to render the box scene, like this: mitsuba -m gpu_autodiff_spectral cbox.xml, I got this image

since mitsuba2 has an OptiX engine as well .then I can use pbrt's OptiX imgtool to denoise it, imgtool denoise-optix cbox.exr --outfile denoised.exr , it yeilds image

it seems the denoiser is not working! these are all I did, I think it should be clear enough, I intend to find out if pbrt's optiX denoising tool can apply to other sources, since the image format is always right, if not, what did I miss that make it work? I think this may has some relation with how optiX's denoiser works, anyway , I think it makes scene to make pbrt's imgtool work on other sources.

jakoaltrd commented 3 years ago

I build pbrt-v4 with ubuntu20.04 in GPU successfully, then i render a scene with 8 spp, the scene is transparent-machines frame542.pbrt, but I changed the fov from 12 to 4, then I got this in 2.1s. ubnuntu20.04 , OptiX7.3, CUDA 11.4, cmake 3.17.1, RTX 3070 2

then I use imgtool to denoise it , it yeilds 1

at last I use 1024 spp to render it, after 237s, it is like 3 then I compared this with the teaser in the official build teaser-transparent-machines it seems that the teaser looks better, maybe. but why the denoiser of OptiX can't work well on transparent things, or this results from lacking of other info like albedo, normals etc. is there a better way to optimize this?

FilipeXavierGama commented 3 years ago

it is common to use Optix's denoiser to denoise a img with low spps, since mitsuba2 also use optix to render, it should be able to use pbrt's optix denoiser imgtool to denoise the output with low spps, so I made a experiment:

I use mitusba2 render a rgb image, then I use imgtool to denoise it, but the output looks the same as the input, it seems pbrt's optix denoiser can't work with external sources? why is that? output: image input: image

In addition to RGB information, the input .exr images must have additional channels holding at least albedo and normals information in order optiX-denoiser take effect. These channels also should have the following names: For RGB channels: {"R", "G", "B"} For Albedo channels: {"Albedo.R", "Albedo.G", "Albedo.B"} For Normal channels {"Nsx", "Nsy", "Nsz"}

PBRTv4 imgtool-optix-denoiser works on any .exr image as long as you meet these requirements. I also have shared my code in some issue related to imgtool-optix-denoiser (please take a look at the list of issues) which avoids some memory leak (in case authors have not fixed this yet). It works fine for me!

If you are not satisfied with the output results, then, the problem is in the optix-denoiser itself which is a learned-based method from Nvidia. Thus, PBRT4 imgtool-optix-denoiser cannot provide better results. The only thing PBRT4 imgtool-optix-denoiser could provide is more flexibility and control over the hyperparameters of optix-denoiser (e.g., blending threshold) and use temporal information (optical flow) in optix-denoiser to denoise sequence of images.

jakoaltrd commented 3 years ago

thanks,it seems my image lacks some albedo and normal channels. if imgtool-optix-denoiser use temporal info, then should I feed it with sequence of images, and does pbrt-v4 docs have detailed specifications.

FilipeXavierGama commented 3 years ago

thanks,it seems my image lacks some albedo and normal channels. if imgtool-optix-denoiser use temporal info, then should I feed it with sequence of images, and does pbrt-v4 docs have detailed specifications.

Optix-denoiser (the tool from Nvidia) supports temporal denoising. However, PBRTv4 imgtool-optix-denoiser does not support this feature, at least not in the current implementation.

mmp commented 3 years ago

does pbrt-v4 docs have detailed specifications.

There is effectively no documentation for pbrt-v4 besides the code (and there won't be for a few more months, until the book is closer to finished.) In the meantime, if you're interested in using it, your best bet is to read the code to figure it out. ;-)