[2023.02.13] How can I understand 3D Vision?

[sghong977]

I just started to try NeRF these days but actually, it doesn't come to my mind how it works. I'm even not familiar with common keywords in this area. So I decided to browse wide and shallow pieces of knowledge of 3D vision.

I'll take this step-by-step process to briefly get to know about this area:

• [x] Collect useful contents
• [ ] Make my own curriculums
• [ ] then absorb them!

---

Useful Contents

3D Vision Lecture
Link: https://www.youtube.com/watch?v=S1_nCdLUQQ8 This is 1 hour of lecture about:

• ways for incorporating 3D structure into neural networks
• representations for 3D data: depth maps, voxel grids, implicit functions, point clouds, and triangle meshes (+GCN)
• metrics for comparing 3D shapes: IoU, Chamfer distance, F1 scores
• camera systems, specifically the difference between canonical and view-centric coordinate systems
• 3D shape prediction: ShapeNet and Pix3D, Mesh R-CNN (for joint object detection & 3D shape prediction)
• But this lecture doesn't cover these topics:

The Other Useful Links

• "An introduction towards 3D Computer Vision" (2018) https://medium.com/@jianshi_94445/an-introduction-towards-3d-computer-vision-71be8ce11956
• This is a course for this year (Spring, 2023), including NeRF. Slides are available. I think it would be helpful to know what is needed to deeply understand NeRFs. https://learning3d.github.io/
• other lectures: https://web.stanford.edu/class/cs231a/syllabus.html, https://www.youtube.com/watch?v=xeWrxMHId88&list=PLSpnHWTONcJ2jhuzWUCH9UXMyJt2drGKa (korean captions)
• Multiview Geometry Book: https://www.robots.ox.ac.uk/~vgg/hzbook/

[sghong977]

Pieces of knowledge

3D Data

mesh

Metrics

Camera

Dataset

using (2d) recognition

---

UV map? Zbrush, Maya, Blender, and many other 3D modeling-related tools use a UV map. What is it? (from the homepage of Blender saying) A UV map is the flat surface representation of a 3D model used to wrap textures. It describes what part of the texture should be attached to each polygon in the model. Each polygon's vertex gets assigned to 2D coordinates that define which part of the image gets mapped. These 2D coordinates are called UVs (compare this to the XYZ coordinates in 3D).

[sghong977]

My Own Curriculum?

I referred to the syllabus here https://learning3d.github.io/spring22/pages/schedule.html.

But video lecture is not available, so I'll watch this https://www.youtube.com/watch?v=xeWrxMHId88&list=PLSpnHWTONcJ2jhuzWUCH9UXMyJt2drGKa first and check whether I got enough knowledge for understanding NeRF by browsing slides of learning3d lecture.

---

Things that I should get are:

• ...