oom when meeting large dataset

FelSiq / DBCV

Efficient implementation in Python of Density-Based Clustering Validation (DBCV) metric, fully compatible with the original MATLAB implementation.

MIT License

11 stars 5 forks source link

oom when meeting large dataset #5

Open JeffLee1874 opened 3 months ago

JeffLee1874 commented 3 months ago

I have a dataset containing 5million samples with 5-dim features (shape: [5m, 5]). I run it on my machine (16core, 120g ram), but oom occur. How can I do to calculate DBCV score in such a large scale?