Question for the rendering

[SYSUykLin]

I saw two rendering method in the code. 1) visibility, hitting probability -> IBRNet -> density, color -> alpha -> hitting probability -> images 2) visibility -> alpha -> hitting probability -> images

However, in the follow code: alpha_values, visibility, hit_prob = self.dist_decoder.compute_prob( prj_dict['depth'].squeeze(-1), que_dists.unsqueeze(0), prj_mean, prj_var, prj_vis, prj_aw, True, ref_imgs_info['depth_range']) we can obtain hitting probability, can we direct render images via hit_prob? Thanks

[liuyuan-pal]

Hi, https://github.com/liuyuan-pal/NeuRay/blob/939af16b5e27bcda4f011d1e43cb833c1508622a/network/renderer.py#L115 this function means we directly render images using the probabilities, which are stated in the supplementary material of NeuRay. The quality for this direct rendering is much lower than IBRNet.

[SYSUykLin]

Thanks for your reply.

I met another question. When I set the "num_worker" of the Dataloader > 2, the error "DataLoader worker (pid 64987) is killed by signal: Killed." is occured. Have you encountered this problem before？

I run the NeuRay model on OmniObject3D dataset

[liuyuan-pal]

Hi, I wrote the training framework by myself at that time and the implementation was somehow very naive which does not support multiple workers for the dataloader. Maybe, you need to switch to a framework like pytorch_lightining. Sorry for the inconvenience.

https://github.com/liuyuan-pal/NeuRay/blob/939af16b5e27bcda4f011d1e43cb833c1508622a/train/trainer.py#L38

[SYSUykLin]

谢谢您的回答，我犯了一个傻逼错误。

另外我想请问一下，大哥你们之前不是有一个omniObject3D的比赛嘛，现在比赛已经结束，你们会把test dataset的GT公布吗？我想用你们比赛用的test dataset进行测试。谢谢谢谢

[liuyuan-pal]

噢噢，这个是NTU ziwei老师那边举办的，不是我这边举办的，我不太清楚这个testset是否会放出来鸭

[SYSUykLin]

谢谢回答，非常感谢

[SYSUykLin]

hello，我还想再请教一个问题，NeuRay一开始是需要用depth map或者cost volume初始化对吧，那您用cost volume初始化的时候有没有使用depth loss呢？因为我这边没有使用depth loss，然后出现了梯度爆炸的问题，谢谢

[liuyuan-pal]

hi，这里在训练的时候是有用depth loss的。不过梯度爆炸可能不一定是depth loss的问题，你可能可以尝试一下不同的lr。

[SYSUykLin]

谢谢您的回复，我看了一下感觉是这个IBRNet输出rgb的问题，我在IBRNet输出这块加了sigmoid。还有大佬我想问一下，为何要加depth loss呢？是不加depth loss就train不了吗？我感觉这么大的数据量应该是足够训练的吧？