aphalo / photobiology

Package ‘photobiology’ defines a system of classes for storing spectral data and accompanying methods and operators. This is the core of a suite of R packages for photobiological calculations.
4 stars 1 forks source link

ciev2.spct #26

Closed CdeMills closed 2 weeks ago

CdeMills commented 6 months ago

Hello

the graph obtained as

autoplot(ciev2.spct)

looks "strange" to me. The maximum is at 556 nm, there are a few bumps in the curve; one around 460 nm, the other at 570 nm.

The docs says "Original data from http://www.cvrl.org/ downloaded on 2014-04-29". But cvrl contains many data.

OTOH, the official data are available from https://cie.co.at/datatable/cie-1931-colour-matching-functions-2-degree-observer inside the file CIE_xyz_1931_2deg.csv

Those data can be used as

myciev2.spct <- as_tibble(read.table("CIE_xyz_1931_2deg.csv", sep=","))
colnames(myciev2.spct) <- c("w.length", "xbar", "ybar", "zbar")
myciev2.spct <- source_spct(w.length=myciev2.spct$w.length,
                            s.e.irrad=myciev2.spct$ybar)
myciev2.spct <- normalize(myciev2.spct, norm="max")
allspecs <- source_mspct(list(CIEv2=myciev2.spct, 
                                          photobio=source_spct(w.length=ciev2.spct$w.length,
                                                                               s.e.irrad=ciev2.spct$s.e.response)))
autoplot(allspecs, annotations=list(c("color.guide", "peaks")))

I used "source_spct" as I wished to compare those weighting functions to LED spectra. The intent is to show the differences between the cvrl and CIE official tables.

Could you please double check ?

Regards

Pascal

aphalo commented 6 months ago

@CdeMills I will check the file I downloaded from the site and compare it against the source you mention, and then check the script that I use to add it to the package.

aphalo commented 6 months ago

@CdeMills

Dear Pascal,

The file I am using for ciev2.spct is "linCIE2008v2e_1.csv", so it is not the 1930 definition. Could this explain the difference? I agree that the bump is surprising, and I will check with an alternative source of the file. I have in most cases also added attributes to the objects conatining metadata to help trace the origin of the data, as can be seen in the example code below. I hope this helps for now.

library(photobiology)
#> News at https://www.r4photobiology.info/
what_measured(ciev2.spct)
#> [1] "CIE 2008 2 degrees V"
comment(ciev2.spct)
#> [1] "CIE 2008 2 degrees V from linCIE2008v2e_1.csv"

Created on 2024-02-26 with reprex v2.1.0

aphalo commented 6 months ago

At the www.cvrl.org site these data are described as CIE (2008) physiologically-relevant 2-deg V(λ) luminous efficiency functions and when plotting them in the web page they show the same bump. It is possible that there is something wrong in the file they are serving at the web site... This is not my field of expertise, so I have assumed that the newer CIE standard was the most relevant. I could still add the 1930 one if it is in widespread use.

aphalo commented 6 months ago

Note to myself: data files can be found at CIE https://cie.co.at/data-tables

CdeMills commented 6 months ago

linCIE2008v2e_1.csv

Dear Pedro I think I spotted the issue. On the CVRL web site, when you arrive at the "Luminous efficiency" page, you have this interface with a few buttons to choose how to display data. If you click on "i", you can read:

"Based on the linear combination of the Stockman and Sharpe (2000) M- and L-cone spectral sensitivities that best fits experimentally-determined 25-Hz, 2° diameter, heterochromatic (minimum) flicker photometric data obtained from 40 observers (35 males, 5 females) of known genotype, 22 with the serine variant L(ser180), 16 with the alanine L(ala180) variant, and 2 with both variants of the L-cone photopigment. The matches, from 425 to 675 nm in 5-nm steps, were made on a 3 log troland xenon white (correlated color temperature of 5586 K, but tritanopically metameric with CIE D65 standard daylight for the Stockman & Sharpe (2000) L- and M-cone fundamentals) adapting field of 16° angular subtense, relative to a 560 nm standard. Both the reference standard and test lights were kept near flicker threshold so that, in the region of the targets, the total retinal illuminance averaged 3.19 log trolands. The new function is extrapolated to wavelengths shorter than 425 nm and longer than 675 nm using the Stockman & Sharpe (2000) cone fundamentals. See Sharpe, Stockman, Jagla & Jägle (2005).

NOTE THAT THE CIE FUNCTIONS ARE CORRECTED VERSIONS OF THE SHARPE et al. 2005 FUNCTIONS. The correction, which is described in Stockman, Jägle, Pirzer & Sharpe (2008), takes into account the fact that the targets used to measure the flicker photometric matches change the adapting chromaticity. See also CIE (2006). The coefficients are now defined to 8 dp for consistency with the CIE physiologically-relevant LMS to XYZ transformation. "

Meaning the file "linCIE2008v2e_1.csv" seems to use the "Sharpe 2005" data; while the one distributed by the CIE is a correction from 2008. I would consider the later as the "right" V(lambda) source.

Regards

Pascal

aphalo commented 6 months ago

Dear Pascal, I will replace the data before the next release. As it happens, a new version of 'photobiology' was accepted one day ago by CRAN, it will take some time before I submit a new version to CRAN. An alternative could be to add a function to 'photobiologyInOut' that imports the data directly from the CIE site, as they seem to be providing data in a well defined format and with metadata as JSON. Best wishes, and thanks, Pedro.

CdeMills commented 6 months ago

Dear Pedro,

"Use the source, Luke ! " (Obi-Wan).

Importing the data from the CIE should be done once in a while, when you upgrade your package. My guess is that having some toolbox performing a download at each startup is not a good idea.

Regards

Pascal

aphalo commented 6 months ago

Dear Pascal,

Yes, you are right. I did not mean at package startup, but rather to add a function in 'photobiologyInOut' for users to import any of the data available as CSV + JSON file pairs at the CIE website. Then they could keep a local copy either of the CIE files or of the R objects in .rda or .rds files. With respect to the subset of data currently in the 'photobiology' package, yes I will update those for the next release, in the same format as they are now available.

Best wishes,

Pedro.

aphalo commented 5 months ago

Version 0.11.2 ended being an off-schedule bug-fix release. New milestone is the next feature update.

aphalo commented 2 weeks ago

For the time being, to preserve backwards compatinility, I added a note to the documentation of these data recommending to download the official files from CIE's website. I do not know at this time if redistribution is allowed or not.