Unit 8 - 3D Vision, Scene Rendering and Reconstruction: Draft outline

[jfozard]

Hi. This is a really deep subject, with much more potential material than can be covered easily by a small team. As a result, I've adapted work by @CyWiz57 to give a minimal outline covering some of the current hot topics - view synthesis, NeRFs, and Gaussian splatting.

Introduction:

• Overview of 3D Vision
• Brief history of 3D Vision
• 3D Vision applications

Terminologies and Basics:

• Linear algebra and transformations.
• Camera models: Pinhole and lens distortion models.
• Representations: Point clouds, meshes, implicit surfaces, volumetric data.

3D Vision:

• Depth estimation from single images (MiDaS)
  • Structure from Motion - Estimating camera positions from multiple images (COLMAP, SuperGlue)
  • Multiview stereo (COLMAP)
  • Volume Rendering
  • Neural Radiance Fields (NeRF)
  • Rendering Signed Distance Fields (NeuS)
  • Sparse View 3D and View Synthesis (PixelNeRF, Zero123)
  • Neural Light Fields (Light Field Networks)
  • Point Cloud Processing (PointNet)
  • Generative 3D Models (Point-E, DreamFusion, Magic3D)
  • Gaussian Splatting
  • Datasets (ShapeNet, Objaverse)

Team members @jfozard @julien-blanchon @CyWiz57 - possibly also Luke, sanae, hwaseem04.

Please let me know if there are things missing, or that can be cut easily.

[johko]

Hey @jfozard, sounds pretty cool to me :hugs:

I'm not an expert though, mainly worked with point clouds in an automotive context. But from a potentical course candidate point of view, the topics seem really interesting. Maybe you can structure the 3D Vision section a bit more (e.g. sub chapters), but apart from that it looks really good :slightly_smiling_face:

[ATaylorAerospace]

@jfozard This will be a great chapter!

I thought of two possible other hot topics under 3D vision if not too much material: :-)

• Real-Time 3D Vision: Could include discussion on-

• [ ] Real-Time Depth Sensing Technologies: Discuss different technologies like Time-of-Flight cameras and their apps.

• [ ] Challenges and Solutions: Talk about the computational and memory challenges in real-time 3D vision.

• Photogrammetry and 3D Mapping: Could include discussion on-

• [ ] Techniques in Aerial and Drone-based Photogrammetry: Its use in 3D mapping and surveying for Agriculture or Real Estate.

• [ ] Integration with GIS: For environmental, farming and urban planning applications.