so-wise / weddell_gyre_clusters

Unsupervised classification of Weddell Gyre profiles
MIT License
2 stars 1 forks source link

Add the silhouette score to the BIC calculations #104

Closed DaniJonesOcean closed 2 years ago

DaniJonesOcean commented 2 years ago

This seemed to help in the ozone case

DaniJonesOcean commented 2 years ago

image

DaniJonesOcean commented 2 years ago

Done! What's interesting here is that the silhouette score is getting worse with higher K. So we're getting more and more overlap, despite an apparent increase in likelihood. So here we see the tradeoff: if we increase K, we get a more sophisticated model with slightly better likelihood but worse overlap. It seems like we can use this to further justify our choice of K=4.

If we use K=2, we have the least amount of overlap between the clusters, but we aren't near the likelihood minimum.

Okay, good!

DaniJonesOcean commented 2 years ago

That being said, sometimes K=3 shows up as the inflection point, despite having quantified the variability as well...could we go for that instead? Maybe it would remove the ambiguity between the transition and the ventilated classes. Could be worth a try

DaniJonesOcean commented 2 years ago

Thought: the process of transforming from one type of profile to another involves changes at the surface, advection, diffusion, and isopycnal changes as well

Picture the profile evolving as it moves - different parts of the profile will change at different rates. Some parts getting warmer, some parts getting colder, etc.

DaniJonesOcean commented 2 years ago

I'm getting much more comfortable with a K=3 model...

image

DaniJonesOcean commented 2 years ago

Done