kan-qi / UMLx

Analyze software architecture and generate insights about software development complexities, risks, and costs.
3 stars 0 forks source link

Evaluate the effect size of out-of-sample accuracy #623

Open kan-qi opened 5 years ago

kan-qi commented 5 years ago

For the comparisons between the models in terms of their out-of-sample accuracy, let's also evaluate the effect size.

The measure we use is called cohen's d. Here is the method of calculating cohen's d.

image

Here is the current implementation.

image

Make sure the above implementation work well on our dataset.