search

YinLiLin / KAML

:bicyclist: Kinship Adjusted Multi-Loci Best Linear Unbiased Prediction
GNU General Public License v3.0
41 stars 12 forks source link

How can I assess the predictive ability of my model? #14

Closed qizhengyang2017 closed 2 years ago

qizhengyang2017 commented 2 years ago

Hi, First of all, thank you for developing the wonderful software. I used all my samples as training data to train a predictive model and the final output is listed below. I want to know the prediction accuracy of the model I got, which is a QTN+K model. It seems I can use Cross-validation to assess my model, but I didn't know how to do that. Is there a parameter in KAML to do that, or I need to do it by myself?

List of 12
 $ y       : num [1:376, 1] 27.9 29.6 29.7 28.5 29.6 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : NULL
  .. ..$ : chr "FL"
 $ beta    : num 29.3
 $ gebv    : num [1:376, 1] -1.478 0.248 0.318 -0.859 0.276 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : NULL
  .. ..$ : chr "FL"
 $ vg      : num [1, 1] 1.09
 $ ve      : num 1.07
 $ qtn     : num [1:2] 1909343 1234880
 $ qtn.eff : num [1:2] -0.383 -0.471
 $ model   : chr "QTN+K"
 $ weight  : num [1:2658921] 0.987 0.987 0.987 0.987 0.987 ...
 $ top.perc: num 0.000923
 $ logx    : num 1.11
 $ K       : num [1:376, 1:376] 0.8634 0.3642 0.0102 -0.0312 -0.1829 ...
YinLiLin commented 2 years ago

Thank you for trying KAML. There is no automatic procedure to implement cross-validation in KAML, it should be done by an additional self-made script.

qizhengyang2017 commented 2 years ago

Thank your for you reply.

I have another question. I noticed that in your example data, testing individuals and training individuals were put together, so after training procedure, the gEBV of testing individual could also be output.

However, in my situation, I want to get a prediction equation and then use this equation to predict the individuals. I am a beginner to the LMM, so I am not sure the equation is like, although I think the final output gives enought information. I suppose the weight vector and the K matrix is needed for prediction, since the format of BLUP model is 𝐲 = 𝐗𝛃 + 𝐙𝐚 + 𝐞. Or maybe just weight vector is needed. my question is:

Suppose I have an individual and its genotype is (1,0,1,2...), how can I get the predicted value by using the KAML output?

Sorry to bother you again.

YinLiLin commented 2 years ago

Sorry that i am not very clear about your question, if you wanna predict any individual, just merge it in the genotype data, with a phenotypic value marked as NA, then it will be predicted automatically by KAML.

qizhengyang2017 commented 2 years ago

Thanks!