search

nightscout / cgm-remote-monitor

nightscout web monitor
GNU Affero General Public License v3.0
2.39k stars 71.63k forks source link

Compact Atlas collections from Admin Tools #7117

Closed goldendel closed 3 months ago

goldendel commented 2 years ago

Is your feature request related to a problem? Please describe. The index on the entries collection is 10x the DB size after removing old entries. Initially it shrunk a little, but after 2 weeks, it's still far bigger than the DB.

Describe the solution you'd like Atlas has a Compact Method available via API. It used to be possible to compact the DB via mlab. There is no easy way to do this other than API having moved to Atlas. It would be helpful to be able to shrink DB size after deleting DB documents via the NIghtscout Admin Tools.

Describe alternatives you've considered I've tried installing MongoDB compass, but it's not clear how to issue the correct command to safely compact the collection.

Additional context image

thedewi commented 8 months ago

Did you ever solve this? I've just spent more than 4 hours trying to understand this mess, and gotten nowhere. None of the examples I can find seem to be applicable to Atlas M0.

goldendel commented 8 months ago

Hi @thedewi. Thanks for picking this up. Not direct solution via this route.

To be quite honest, I'd forgotten I raised this, given it was back in September 2021.

With the move to paid plans at Heroku, I moved to a NightScout as a service provider (NightScout Pro). There, the disk space is pretty much unlimited, so it's less of an impact. Being able to compact the DB would be more space efficient, but I think there are likely higher value things you can work on.

I've dropped a note to Andy Low from NightScout Pro to see if he has strong opinions on this. If not, I'm happy to close the issue/feature request.

thedewi commented 8 months ago

Thanks. I'm not sure a nice solution is possible on the Mongo Atlas free tier, but I did it the slow disruptive way - told Nightscout to delete old records, did a full mongodump, terminated the whole cluster, and did a full mongorestore onto a new one. The restore part takes an hour or more; I guess rebuilding every index with minimal CPU time is a big job. But ultimately it coped better than Northflank's free tier, which seems to have more disk space, but actually runs out of memory and bricks itself if you try to rebuild big indexes 😲

goldendel commented 3 months ago

Closed as more compact DB size is less of an issue with NSaaS. Given the dev community have not picked this up in almost 3 years they clearly have other priorities for Nightscout.