search

PJLab-ADG / neuralsim

neuralsim: 3D surface reconstruction and simulation based on 3D neural rendering.
MIT License
590 stars 31 forks source link

How do you handle dynamic objects in training? #2

Closed LMerCy closed 1 year ago

LMerCy commented 1 year ago

It seems you are trying to evaluate your model in scenes with little dynamic objects, but how do you handle dynamic object in training?

ventusff commented 1 year ago

In this paper, we manually select sequences with almost no dynamic objects. Even so, there will still be some dynamic pedestrians left (for example in seg153495 there are two pedestrians standing on the right side of the street), and in this case we use the "people" mask inferred using SegFormer to further ignore them in pixel sampling and pixel loss, following BlockNeRF's solution.

But this is a rare case, most of the selected (and trimmed as mentioned in supp) sequences contain literally no significant dynamc objects.

This is just a consideration of addressing issues one by one. We will deal with multiple-object reconstruction in a very soon upcoming work.

LMerCy commented 1 year ago

Great, looking foward to your future work!