This is a draft pull request for fast emitters and integrators that work with regular grids, It was originally proposed for Cherab (issue #204) but, as @CnlPepper suggested, these emitters and integrators are more suitable for Raysect.
I tried to address all the issues raised in the discussion.
The emitters work as follows:
The emission can be provided as a 4D array, 2D array or scipy sparse matrix along with array of wavelengths for which this emission is specified. The units are: W / (m^3 str nm) for continuous spectrum and W / (m^3 str) for discrete spectrum (spectral lines). The user specifies the type of spectrum with continuous argument, which is True by default. RegularGridEmitter stores the emission as a compressed sparse column-major matrix (scipy.sparse.csc_matrix) for fast column access required for resampling.
Similar to SpectralFunction, the RegularGridEmitter automatically rebuilds the cache when the existing cache is not match the ray spectral properties. The cache validity is checked each time the emission_function() or RegularGridIntegrator.integrate() is called. The cache is stored as a compressed sparse row-major matrix (scipy.sparse.csr_matrix) for fast row access required for volume integration. If continuous=True, the resampling is performed the same way as in InterpoatedSF, but with optional extrapolation. If continuous=False, the spectrum is not interpolated, instead the emission at each spectral bin sums the emissions at all spectral lines that fall into that bin, divided by the bin's width. Thus, the cache always contain the emission in W / (m^3 str nm).
Since RegularGridEmitter is designed to work with very large grids, it has a few memory-saving options.
If the emission is provided in float32, the cache will be also stored in float32. Also, the user has an option to store the cache in float32 even if the emission is in float64.
In multi-process rendering, each process creates it's own cache. This looks like a correct behaviour, but can easily "eat" all available memory. To prevent this from happening, the user can preset the cache with cache_build(min_wavelength, max_wavelength, bins) function before camera.observe() is called. All processes will use this cache as long as it remains valid (of course building the cache in advance is useless in dispersive rendering).
The user has an option to override the cache with cache_override(new_cache, min_wavelength, max_wavelength), so if there is not enough memory to store both the original emission and the cache, the user can provide the cache only by initialising the emitter with an empty csc_matrix and then calling cache_override(). Again, this will not work in case of dispersive rendering.
The cache is directly accessible from C-interface through read-only memoryviews. All functions that control the cache are accessible in both Python and C.
Currently, building the cache is slow, and it's noticeable for large grids. I'll try to optimise it in the future.
This draft pull request contains 4 demos. The first one is a kind of tutorial.
The code has built-in documentation but I didn't update the .rst files yet. Please look through the code, and if it's OK in general, I'll update the documentation and make this PR ready for review.
Also, I have a question to @CnlPepper. The SpectralFunction has custom __getstate__(), __setstate__(), __reduce__() methods. Do I need them in RegularGridEmitter too?
I updated the documentation for the API and demonstrations. Regarding the __getstate__() and __setstate__() methods, I think these are not necessary in the material class.
This is a draft pull request for fast emitters and integrators that work with regular grids, It was originally proposed for Cherab (issue #204) but, as @CnlPepper suggested, these emitters and integrators are more suitable for Raysect.
I tried to address all the issues raised in the discussion.
The emitters work as follows:
continuous
argument, which isTrue
by default.RegularGridEmitter
stores the emission as a compressed sparse column-major matrix (scipy.sparse.csc_matrix) for fast column access required for resampling.SpectralFunction
, theRegularGridEmitter
automatically rebuilds the cache when the existing cache is not match the ray spectral properties. The cache validity is checked each time theemission_function()
orRegularGridIntegrator.integrate()
is called. The cache is stored as a compressed sparse row-major matrix (scipy.sparse.csr_matrix) for fast row access required for volume integration. Ifcontinuous=True
, the resampling is performed the same way as inInterpoatedSF
, but with optional extrapolation. Ifcontinuous=False
, the spectrum is not interpolated, instead the emission at each spectral bin sums the emissions at all spectral lines that fall into that bin, divided by the bin's width. Thus, the cache always contain the emission in W / (m^3 str nm).RegularGridEmitter
is designed to work with very large grids, it has a few memory-saving options.cache_build(min_wavelength, max_wavelength, bins)
function beforecamera.observe()
is called. All processes will use this cache as long as it remains valid (of course building the cache in advance is useless in dispersive rendering).cache_override(new_cache, min_wavelength, max_wavelength)
, so if there is not enough memory to store both the original emission and the cache, the user can provide the cache only by initialising the emitter with an empty csc_matrix and then callingcache_override()
. Again, this will not work in case of dispersive rendering.Currently, building the cache is slow, and it's noticeable for large grids. I'll try to optimise it in the future.
This draft pull request contains 4 demos. The first one is a kind of tutorial. The code has built-in documentation but I didn't update the .rst files yet. Please look through the code, and if it's OK in general, I'll update the documentation and make this PR ready for review.
Also, I have a question to @CnlPepper. The
SpectralFunction
has custom__getstate__()
,__setstate__()
,__reduce__()
methods. Do I need them inRegularGridEmitter
too?