hollorol / RBBGCMuso

RBBGCMuso is a software package that supports the application of the Biome-BGCMuSo biogeochemical model.
GNU General Public License v2.0
20 stars 11 forks source link

How to interpret the relative importance of sensitivity analysis results #16

Closed 857687671 closed 1 year ago

857687671 commented 3 years ago

Hello, I would like to know whether there are some criteria to judge the relative importance of sensitivity analysis results. that is, if the value of the analysis results is greater than a certain value, it can be judged that the parameter is a sensitivity parameter. For example, if the analysis result is more than 10%, it can be determined that the parameter is sensitive, and the next step of parameter optimization process is needed. Thank you.

hollorol commented 3 years ago

Hello, This is one weakness of these kind of methods. The threshold is always subjective. As a rule of thumb I would take into account that the calibration will be longer if you choose more parameters. If I am able to run a calibration with 10^6 iteration, I would say 6-10 parameter is the maximum I can calibrate; therefore I would select the top 6 most important variables. Another method for choosing is to select the minimum percentage of variance you are Ok with. For example If I have 10 parameters and the top 4 of them account for 90% of variability, I can drop the rest because they have little impact, and I throw away only 10 percent of the total variance. I will implement a method in the near future which is less subjective (most of the feature selection methods are subjective). Note that you should consider using multiple output variables for sensitivity analysis and use the results together to find relevant parameters.

857687671 commented 3 years ago

Hello, thanks for your detailed explanation, which is very helpful to me. Looking forward to the release of your new method.

------------------ 原始邮件 ------------------ 发件人: "hollorol/RBBGCMuso" @.>; 发送时间: 2021年6月22日(星期二) 下午3:03 @.>; @.**@.>; 主题: Re: [hollorol/RBBGCMuso] How to interpret the relative importance of sensitivity analysis results (#16)

Hello,

This is one weakness of these kind of methods. The threshold is subjective. As a rule of thumb I would take into account that the calibration will be longer if you chose more parameter. If I am able to run a calibration with 10^6 iteration, I would say 6-10 parameter is the maximum I can calibrate; therefore I would select the top 6 most important variable. Another method for choose is to select the minimum percentage of variance you are Ok with. For example If I have 10 parameters and the top 4 of them account for 90% of variables, I can leave the rest because they have little impact, and I throw away only 10 percent of the total variance.

I'll will implement a method which is less subjective (most of the feature selection methods are subjective).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.