kan-qi / UMLx

Analyze software architecture and generate insights about software development complexities, risks, and costs.
3 stars 0 forks source link

Rank the models based on their accuracy evaluation and goodness of fit. #618

Open kan-qi opened 5 years ago

kan-qi commented 5 years ago

Rank the models based on their accuracy evaluation and goodness of fit:

  1. check the goodness of fit evaluation results for the compared models:

image

  1. check the estimation accuracy result (cross-validation results): image

  2. put some code to order the models in terms of the above evaluation results: image

The ordering criteria:

Estimation accuracy measures:

'mmre': the smaller the better pred15': the larger the better 'pred25': larger -> better 'pred50': larger -> better "mdmre": smaller -> better "mae": smaller -> better

Goodness of fit measures: R2: The larger the better Eta-squared: larger -> better