search

ubergeek77 / Lemmy-Easy-Deploy

Deploy Lemmy the easy way!
MIT License
133 stars 14 forks source link

Pict-rs size #50

Closed calr0x closed 1 year ago

calr0x commented 1 year ago

This is not directly related to you but after significant amount of research I couldn't find any answer so I thought I would hit on you.

My 80 gig VPS filled up with about 45 gig of images and Lemmy died.

I think this is going to be a problem running the small instance for myself I was trying to run and I was wondering if you had any idea on the right way to prune it. My original assumption is that we would not be storing the images locally but it appears we are.

I shut down the instance and just deleted the two folders in the "files" dir. When I started the instance back up it had some issue that prevented it from running.

I didn't investigate too much as I'm on vacation so I just nuked the instance for now as I have other services that need some space.

Let me know if you're seeing this too!

ubergeek77 commented 1 year ago

Unfortunately I can't be of much help for pict-rs pruning. I have not been able to find information on this either. The general community stance seems to be "Lemmy has high disk space requirements, increase your disk space."

The closest thing is a new feature in pict-rs 0.4 that allows you to use object storage (S3) for the pict-rs data. I don't support object storage automatically, but it only involves setting some environment variables, which I do support. You will have to manage the migration on your own manually, however. Though it may be easier to just start over using object storage from the beginning.

calr0x commented 1 year ago

At the rate it filled up I don't see object storage being in order of magnitude lower and I think this rules out running a local instance for now.

Thanks for the input!

On Mon, Jul 24, 2023, 11:39 AM ubergeek77 @.***> wrote:

Unfortunately I can't be of much help for pict-rs pruning. I have not been able to find information on this either. The general community stance seems to be "Lemmy has high disk space requirements, increase your disk space."

The closest thing is a new feature in pict-rs 0.4 that allows you to use object storage (S3) for the pict-rs data. I don't support object storage automatically, but it only involves setting some environment variables, which I do support. You will have to manage the migration on your own manually, however. Though it may be easier to just start over using object storage from the beginning.

— Reply to this email directly, view it on GitHub https://github.com/ubergeek77/Lemmy-Easy-Deploy/issues/50#issuecomment-1648158039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMRZNQB3TJGWBRHBOLJYQN3XR2JKZANCNFSM6AAAAAA2USPX4I . You are receiving this because you authored the thread.Message ID: @.***>

ubergeek77 commented 1 year ago

You're welcome!

For what it's worth, I am running my own instance just fine on a VPS with only 50GB of total space. Pict-rs is filling up with thumbnail data, but it's at less than 10GB, I've been running for over a month, and I've never pruned it.

If you are seeing 45GB, either a significant number of images are being uploaded to your instance by users, or you federate with an order of magnitude more instances than I do, causing many, many more thumbnails to be cached.

But for a use case of just a single user instance, you generally should not be seeing anywhere near the amount of image data you quoted.

calr0x commented 1 year ago

Hmm ok. I'll try and rebuild and see!

On Mon, Jul 24, 2023, 11:46 AM ubergeek77 @.***> wrote:

You're welcome!

For what it's worth, I am running my own instance just fine on a VPS with only 50GB of total space. Pict-rs is filling up with thumbnail data, but it's at less than 10GB, I've been running for over a month, and I've never pruned it.

If you are seeing 45GB, either a significant number of images are being uploaded to your instance by users, or you federate with an order of magnitude more instances than I do, causing many, many more thumbnails to be cached.

But for a use case of just a single user instance, you generally should not be seeing anywhere near the amount of image data you quoted.

— Reply to this email directly, view it on GitHub https://github.com/ubergeek77/Lemmy-Easy-Deploy/issues/50#issuecomment-1648169017, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMRZNQA34VZQ4NQJGDKR3I3XR2KEDANCNFSM6AAAAAA2USPX4I . You are receiving this because you authored the thread.Message ID: @.***>