gengshan-y
commented
6 months ago Hi @GloryyrolG, there are quite a few differences between banmo and viser. We also compared the two in the paper (e.g., Fig 2, Fig. 4, Tab. 1 and Tab.2 here). Thinking back from my own view, the crucial bits are,
- viser uses differentiable rasterization with meshes, which was difficult to optimize at the time due to self-intersection issues; banmo does not suffer from that thanks to a more flexible representation (neural sdf+volume rendering)
- viser learns long-range correspondence from scratch with contrastive losses. We found the joint optimization to be quite challenging sometimes (more in the limitation section of viser); In banmo, we find using a pre-trained feature extractor (CSE) + pre-training a pose network makes the optimization much more robust
There are also other improvements such as delta skinning weights, re-boning, active sampling, etc. All those small deltas pushed it through the last mile.
GloryyrolG
Hi @gengshan-y @jhugestar et al.,
May I ask the essential difference between the work & ur previous viser work?
but most losses & pipeline look quite similar to me... any help would be thx & best,
for 1. banmo & viser both have;