anl13
commented
5 months ago Thanks for the open source code, I have been following your team's research for a long time. However, my research is on underwater fish reconstruction, can this work generalize to fish-shaped animals？Can you give me some advice
Best wishes
Thank you for your attention! Definitely you can apply similar methods (multi-view capture + mesh fitting) to fishes, and it is even better for fish than pig/mouse to apply a shape model to capture the motion. Previously, I noticed that there are some papers about 3D tracking of freely-moving zebra-fish. However, papers on social shape tracking of fishes seem lacking. You can follow some human pose estimation work written in python (honestly python projects are easier to follow) to adapt to fish cases. Fish reconstruction is excerllent!
More technically, fish has less articulated shape, and the shape variance are not that much. One way is designing a mean shape and adding some scale control over the body; Another way is capturing several fishes of the target species using a camera/cameras, and building a shape-blend-shape model similar to SMAL. I think it is practical because there are useful scanning tools now (such as nerf-studio or 3D gaussian splatting). I believe build a shape model of fish can be realized in the near future. About tracking, you may first try some basic principles like MAMMAL or social behavior atlas or MVPose to identify the key challenges of fish tracking.
If you come up with some problems, you may contact me personally. I am willing to help as much as I can. Best wishes.
Thanks for the open source code, I have been following your team's research for a long time. However, my research is on underwater fish reconstruction, can this work generalize to fish-shaped animals？Can you give me some advice
Best wishes