BANMo VS. ViSER

[GloryyrolG]

Hi @gengshan-y @jhugestar et al.,

May I ask the essential difference between the work & ur previous viser work?

1. is the use of cse.
2. an additional 3d cycle consistency.

but most losses & pipeline look quite similar to me... any help would be thx & best,

for 1. banmo & viser both have;

2. banmo uses cse while viser learns from scratch;
3. they both use nerf/sdf; correct me if i am wrong...

[gengshan-y]

Hi @GloryyrolG, there are quite a few differences between banmo and viser. We also compared the two in the paper (e.g., Fig 2, Fig. 4, Tab. 1 and Tab.2 here). Thinking back from my own view, the crucial bits are,

• viser uses differentiable rasterization with meshes, which was difficult to optimize at the time due to self-intersection issues; banmo does not suffer from that thanks to a more flexible representation (neural sdf+volume rendering)
• viser learns long-range correspondence from scratch with contrastive losses. We found the joint optimization to be quite challenging sometimes (more in the limitation section of viser); In banmo, we find using a pre-trained feature extractor (CSE) + pre-training a pose network makes the optimization much more robust

There are also other improvements such as delta skinning weights, re-boning, active sampling, etc. All those small deltas pushed it through the last mile.

[GloryyrolG]

Thx for the detailed explanation. appreciate its help.