Closed peroveh closed 6 days ago
Hello,
This is not a defect and we probably ought to update the documentation in that regard but you should not be using colour.MultiSpectralDistributions
in that instance but a numpy.NDArray
instead.
Under the hood, colour.MultiSpectralDistributions
uses instances of colour.SpectralDistribution
which in turn is derived from colour.continuous.Signal
class which is an interpolator so that one can sample the spectrum at any arbitrary point. This offers a lot of flexibility for many things but it is, given the above, going to be heavy when using thousands or more spectra because of initialisation time and memory cost.
In the situation where you are, an array is required and you will find out that the colour.msds_to_XYZ
definition actually does work with them! We use it to perform hyperspectral image integration. You certainly lose quite a lot of flexibility in term of manipulation of the spectra but it allows manipulation of much larger datasets.
Cheers,
Thomas
Description
I am working on large sets of multispectral data (Zhang_41 million reflectances) which comes in zips of > 10 million spectra at 31 wavelengths. I thought i could use the MultiSpectral distribution to contain the total spectra from a light, all reflectances, and convert this to XYZ, but it fails already in the initialization of the object if i have higher than 1000 ish vectors.. (3 sec timeout)... Original spectral data are huge files (link below) data= scipy.io.loadmat('Ref5_10_unique_expand.mat') # loads from matlab file #https://www2.cs.sfu.ca/~colour/data/Zhang_41MillionReflectances/ Below i test with a dummy data array of size 31 by 100 and subsequently try 1000 (i need millions)
With size (31,1000).. it takses a lot of cpu, time and gives warnings Should not this datatype large datasets permitting fast vectorized conversion to XYZ for example?
Code for Reproduction
Exception Message
Environment Information