tnc-br / ddf-isoscapes

4 stars 0 forks source link

Generate xgboost using UC Davis data instead of Craig Gordon Model #87

Closed jmogarrio closed 1 year ago

jmogarrio commented 1 year ago

If possible, could we clarify what the real samples are that we're referring to here?

It might also be helpful to specify which one was the Craig Gordon Model data, just for future reference.

jmogarrio commented 1 year ago

Poor results, follow up on recording output, ben will dump output into bug and mark it done.

benwulfe commented 1 year ago

New Best Accuracy 0.4777777777777778 at p_value 0.01 New Best Accuracy 0.5111111111111111 at p_value 0.03 New Best Accuracy 0.5222222222222223 at p_value 0.05

Debugging metrics p-value: 0.05 True Negative: 23 False Positive: 22 True Positive: 24 False Negative: 21 Total rows: 90

Precision d18O cellulose w/ xgboost ucdavis: 0.5217391304347826 Recall d18O cellulose w/ xgboost ucdavis: 0.5333333333333333

benwulfe commented 1 year ago

optimizing for precision yields: Debugging metrics p-value: 0.76 True Negative: 41 False Positive: 4 True Positive: 6 False Negative: 39 Total rows: 90

Precision d18O cellulose w/ xgboost ucdavis: 0.6 Recall d18O cellulose w/ xgboost ucdavis: 0.13333333333333333

with a recall of about 13%

benwulfe commented 1 year ago

We need to re-run the t-test analysis because we have some minor tweaks to make in the t-test code to ensure we are reporting the correct numbers.

benwulfe commented 1 year ago

I believe Luis ran this and presented RMSE to partners.