This PR contains the implementation of a physically-based multi-lens camera created as a bachelor thesis
supervised by @LZaw. It partially covers the topics of the issue "Physically-based camera" #1740 and consists of the complete code as well as test scenes.
The source code is located in src/appleseed/renderer/modeling/camera/multilenscamera.cpp and src/appleseed/renderer/modeling/camera/multilenscamera.h.
The test scenes are located in sandbox/tests/test scenes/multilens camera/.
Lens prescriptions to use with the model are located in sandbox/lenses/.
TODOs
It is currently a draft pull request, as there are some open tasks that will be completed once I submit the thesis:
[x] Improve lens file handling
[x] Make lens file path relative in test scenes
[x] Debug & test connect_vertex()
[x] Presumably some code revisions to comply with the styling conventions
Problem Description
All cameras currently available in appleseed are approximations of real models and sacrifice realistic camera phenomena for performance. There are however scenarios, where images as if shot on a real camera are desired, for example when combining real images with computer-generated footage.
Solution Description
To achieve photorealistic images, a simulation of a real lens with many lens elements is required. In literature, there are mainly two approaches for that, which I like to call "ray simulation" and "polynomial optics". Below, I list some characteristics of the two.
Ray simulation
What: Iterate through the list of lens elements and use Snell's law of refraction to compute the path of the ray.
Pro: Pretty straight-forward, reasonable computation overhead, fits well into the current implementation
Con: performance depends on lens complexity, intelligent sampling for lens points is required to not waste too many rays
Polynomial Optics
What: See lens as a function and polynomially approximate it
Pro: Evaluation of polynomial is fast and not dependent on lens
Con: Building the polynomial takes time, high approximation error can lead to bad quality, high polynomial degree can lead to high precomputations
As discussed, this PR extends the cameras by another one and is located in src/appleseed/renderer/modeling/camera. This PR contains the following changes:
Add MultiLensCamera that inherits from PerspectiveCamera. The most important parts of it are:
Read lens prescription files that the user specifies as a file path. They can either be self-specified or some are currently located in sandbox/lenses/.
Modify lens based on user-specified values (focal length, focus distance, f-number)
Add MultiLensCameraFactory to CameraFactoryRegistrar
Change void spawn_ray(..) to bool spawn_ray(..). This is required, as rays can be blocked by the aperture and should then not contribute to the illumination calculation. This change affects camera.h and all <name>camera.cpp files, as well as src/appleseed/renderer/kernel/rendering/generic/genericsamplerenderer.cpp and src/appleseed/renderer/kernel/rendering/scenepicker.cpp.
Move extract_focal_length(..) from PerspectiveCamera::on_render_begin(..) to all childen (pinhole, thinlens, fisheye). This is because MultiLensCamera has its own focal length extraction that allows for not specifying a focal length.
Realistic bokeh with custom shapes (left: thin lens, right: multi-lens)
Natural vignetting
It cannot produce:
Chromatic aberration: Would require a redesign of the camera and ray implementation, such that the ray carries a wavelength and the camera refracts the rays based on wavelength
Lens flare: Efficient solutions exist, but were out of scope for this thesis.
Performance
I spent quite some time analyzing the performance of the implementation in relation to pinhole and thin lens camera. For details, see Chapter 6.2 of my thesis. The main takeaways were:
The multi-lens camera is slower in all cases, but the overhead is constant. The more complex the scene, the less severe the overhead.
The overhead only depends on the complexity of the lens, the simpler the lens, the closer the rendering time to pinhole and thin lens.
The single exit pupil is usually enough to have a small ray failure rate. Custom aperture shapes, however, can block large portions of rays, leading to better rendering times, but also to under-exposed images.
A big factor that worsens the times are ray differentials, their inclusion triples the overhead. A toggle to switch off ray differentials is therefore included.
Conclusion
The implemented model is not a thin lens replacement, but offers advantages in some scenarios, especially when bokeh or vignetting is important. Although performance is a bit worse than pinhole and thin lens, the overhead is constant and therefore too significant in complex scenes. With simple scenes though, the multi-lens camera is probably not worthwhile to use.
Thanks for the first look!
Completed are all the syntax and style issues.
Still left are things that require a bit more thought and testing (memory/speed improvements, function locations, exception handling, ...)
Summary
This PR contains the implementation of a physically-based multi-lens camera created as a bachelor thesis supervised by @LZaw. It partially covers the topics of the issue "Physically-based camera" #1740 and consists of the complete code as well as test scenes.
src/appleseed/renderer/modeling/camera/multilenscamera.cpp
andsrc/appleseed/renderer/modeling/camera/multilenscamera.h
.sandbox/tests/test scenes/multilens camera/
.sandbox/lenses/
.TODOs
It is currently a draft pull request, as there are some open tasks that will be completed once I submit the thesis:
connect_vertex()
Problem Description
All cameras currently available in appleseed are approximations of real models and sacrifice realistic camera phenomena for performance. There are however scenarios, where images as if shot on a real camera are desired, for example when combining real images with computer-generated footage.
Solution Description
To achieve photorealistic images, a simulation of a real lens with many lens elements is required. In literature, there are mainly two approaches for that, which I like to call "ray simulation" and "polynomial optics". Below, I list some characteristics of the two.
Ray simulation
Polynomial Optics
In the end, my choice fell on ray simulation, more precisely the model described in "Realistic rendering of bokeh effect based on optical aberrations" with elements from "An Accurate and Practical Camera Lens Model for Rendering Realistic Lens Effects" and Physically Based Rendering: From Theory to Implementation. For more details on the the comparison and the choice, please see Chapter 3 of my thesis.
Implementation
As discussed, this PR extends the cameras by another one and is located in
src/appleseed/renderer/modeling/camera
. This PR contains the following changes:MultiLensCamera
that inherits fromPerspectiveCamera
. The most important parts of it are:sandbox/lenses/
.MultiLensCameraFactory
toCameraFactoryRegistrar
void spawn_ray(..)
tobool spawn_ray(..)
. This is required, as rays can be blocked by the aperture and should then not contribute to the illumination calculation. This change affectscamera.h
and all<name>camera.cpp
files, as well assrc/appleseed/renderer/kernel/rendering/generic/genericsamplerenderer.cpp
andsrc/appleseed/renderer/kernel/rendering/scenepicker.cpp
.extract_focal_length(..)
fromPerspectiveCamera::on_render_begin(..)
to all childen (pinhole, thinlens, fisheye). This is becauseMultiLensCamera
has its own focal length extraction that allows for not specifying a focal length.Results
The implemented model can achieve the following:
It cannot produce:
Performance
I spent quite some time analyzing the performance of the implementation in relation to pinhole and thin lens camera. For details, see Chapter 6.2 of my thesis. The main takeaways were:
Conclusion
The implemented model is not a thin lens replacement, but offers advantages in some scenarios, especially when bokeh or vignetting is important. Although performance is a bit worse than pinhole and thin lens, the overhead is constant and therefore too significant in complex scenes. With simple scenes though, the multi-lens camera is probably not worthwhile to use.