so-wise / weddell_gyre_clusters

Unsupervised classification of Weddell Gyre profiles
MIT License
2 stars 1 forks source link

Improve treatment of missing data #2

Closed DaniJonesOcean closed 3 years ago

DaniJonesOcean commented 3 years ago

At present, in the example notebook 1.0, missing data is either interpolated or replaced with a mean value. This is not likely to be the most appropriate method. We should decide how to treat missing data from the entire gyre dataset. For example, the data may be "missing" from certain depths because the profiles are taken from shallow bathymetry. It's not really sensible to replace with mean values below this bathymetry.

I plan to discuss this with collaborators at a meeting later today.

DaniJonesOcean commented 3 years ago

At present, we're using the approach of selecting the depth range and only keeping profiles that cover this entire depth range.

DaniJonesOcean commented 3 years ago

This approach seems fine for now. I'm not sure that scaling the data by the full depth of the bathymetry is a good idea. I'm not sure that makes much sense.