Problem with calculating uncertainty

[pksi4E]

I'm not sure if I understand what is going on with calculating uncertainty using mean and quantile functions. I think it should be done only with sd (like in PKN-ISO/IEC Guide 98-3/Suppl.1, 7.6) or by applying set_errors to each input quantity of the model and then we would get the proper uncertainty by specifying measurement function, or maybe I'm missing something. What do you think?

https://github.com/dawidkucharski/AI_for_surface_metrology/blob/679322516aa5acea77b4a831e4c13c85af1bb297/R_ML.R#L565-L577

[dawidkucharski]

The MC uncertainty evaluation is adopted from Shaw Den, B. (2017). Uncertainty Analysis of Experimental Data with R. Taylor & Francis Group, and in fact, it is according to the ISO/IEC Guide 98-3/Suppl.1, however, by using bootstrap confidence interval - using the quantile() function. Two reasons why it is not by using straight sd or error functions: 1 - there is no specified measurement function (as always with data from surface texture measurements), and it is not the same as error function that may be estimated. 2 - as ISO norm always refers to, the new data after MC simulation are moduled by the erros, so the coverage interval for the output quantity is taken into account. The results are similar to sd but based on quantiles, which is more robust. If You found an error in the Shaw Den, B. (2017) pp. 145, please let us know. BTW., the standards are nicely described for easy measurements like temperature measurements, where the function is well known. We have (as always) some difficulties. Ps. From statistics: the quartiles are connected with mean and standard deviation, so in the code, the standard deviation is calculated as mean-Q.

We are fully open to any new idea of uncertainty evaluation in the code, so fill free to propose.

[pksi4E]

Regarding GUM_TaylorHobson(wzNr6_chropC_TaylorHobson112_1534) measurement, in my opinion measurement function is specified in the line https://github.com/dawidkucharski/AI_for_surface_metrology/blob/8276a005ea3a8bea616064b551bf3b48e2fde727/R_ML.R#L561 which you can compare to the pages 135, 136 in the article [1] Veen, A.M.H.v.d., Cox, M.G. Getting started with uncertainty evaluation using the Monte Carlo method in R (2021), in which the code also looks similar. To apply Monte Carlo method measurement function has to be specified, and sensitivity coefficients are not needed as in TSM method which use law of propagation of uncertainty, so yes set_errors in this case is not appropriate, because it calculates error with TSM method, as I understand.

Again regarding measurement mentioned above, the standard deviation of boot.r is equal 0.03067356 (approx. 30 nanometers) where length(boot.r) equals 1e5, so I think it’s okay as it is in GUM Suppl. 1, but still it is only standard uncertainty. In [1] expanded uncertainty is calculated with (quantile 97.5% - quantile 2.5%)/2 and equals 0.05380422 (approx. 53 nanometers), which is very close to your uncert value, i.e. mean - quantile 2.5% in your code, which equals 0.05388556, but there is no coverage factor, and as in [1] this we get, assuming approximation of the coverage interval with symmetric PDF, by dividing expanded uncertainty and standard uncertainty, which equals with my calculations 1.754091. By the way, this non-parametric bootstrap method looks promising as it is not needed to determine the type of distribution for experimental data, nice.

Nevertheless, the point is on something else, that is how measurement function looks like, and I’m not sure in your code what input quantities refer to exactly, why such values were selected and in what units, and why such PDFs are chosen. It should be better described. Take a look at this DKD Annex A document about uncertainty of measurement in the calibration of the roughness standards we mentioned and/or view e.g. S. Zahwi, M. Koura, A. Mekawi Factors influencing uncertainty evaluation for surface roughness measurements (2003), where in the second also several input quantities are mentioned, but unfortunately nothing about how they should be calculated. As it seems to me right now, the problem with measurement model for surface measurement is that the effort should be made to specify the model properly within reason including influence of, let me say, multiple levels, e.g. see image on page 340 in article M. Tomov, M. Kuzinovski, Piotr Cichosz A New Parameter of Statistic Equality of Sampling Lengths in Surface Roughness Measurement (2013), I mean influence on the level of parameterization, filtering, maybe even fitting, and on the level of traced profile. Some simple examples you can view in Part V in B. Muralikrishnan, J. Raja Computational Surface and Roundness Metrology (2009). Hope this helps.

[dawidkucharski]

Again regarding measurement mentioned above, the standard deviation of boot.r is equal 0.03067356 (approx. 30 nanometers) where length(boot.r) equals 1e5, so I think it’s okay as it is in GUM Suppl. 1, but still it is only standard uncertainty. In [1] expanded uncertainty is calculated with (quantile 97.5% - quantile 2.5%)/2 and equals 0.05380422 (approx. 53 nanometers), which is very close to your uncert value, i.e. mean - quantile 2.5% in your code, which equals 0.05388556, but there is no coverage factor, and as in [1] this we get, assuming approximation of the coverage interval with symmetric PDF, by dividing expanded uncertainty and standard uncertainty, which equals with my calculations 1.754091. By the way, this non-parametric bootstrap method looks promising as it is not needed to determine the type of distribution for experimental data, nice.

Thank, You, so we are on the right track. So, please describe the final unc that you recommend to be used in the code: is by dividing expanded uncertainty and standard uncertainty, which equals this example 1.754091 ?

Nevertheless, the point is on something else, that is how measurement function looks like, and I’m not sure in your code what input quantities refer to exactly, why such values were selected and in what units, and why such PDFs are chosen. It should be better described. Take a look at this DKD Annex A document about uncertainty of measurement in the calibration of the roughness standards we mentioned and/or view e.g. S. Zahwi, M. Koura, A. Mekawi Factors influencing uncertainty evaluation for surface roughness measurements (2003), where in the second also several input quantities are mentioned, but unfortunately nothing about how they should be calculated. As it seems to me right now, the problem with measurement model for surface measurement is that the effort should be made to specify the model properly within reason including influence of, let me say, multiple levels, e.g. see image on page 340 in article M. Tomov, M. Kuzinovski, Piotr Cichosz A New Parameter of Statistic Equality of Sampling Lengths in Surface Roughness Measurement (2013), I mean influence on the level of parameterization, filtering, maybe even fitting, and on the level of traced profile. Some simple examples you can view in Part V in B. Muralikrishnan, J. Raja Computational Surface and Roundness Metrology (2009). Hope this helps.

Thank You for the tips. Yes, the function is random at that moment. I hope that measurement system users will know what the actual measurement function looks like. I asked about that :). I will certainly look at the references as well.