search

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
11.26k stars 747 forks source link

borg2: warning: cloud pricing can be expensive for single-object storage #8431

Open ThomasWaldmann opened 2 months ago

ThomasWaldmann commented 2 months ago

Had a look there: https://www.backblaze.com/cloud-storage/transaction-pricing

If each small file results in 1 chunk and each chunk is stored into a separate cloud object, that would mean 1M small (new) files cost $4 to back up using b2.

Also bad: they are also accounting per "ls" (listing 1 directory) and as current borg2 beta11 uses a nested directory structure of 1+256+65536 directories, just doing one full list to get all object IDs are already more than 64k "ls" calls (borg check and borg compact do that).

So, backblaze b2 is accounting "per file / per key / per object" and "per ls" - and not just per MB/GB/TB storage or traffic.

Is that common, do other providers do that also?

bjo81 commented 2 months ago

Other cloud storage providers like https://www.idrive.com/s3-storage-e2/ or https://wasabi.com/pricing don't charge for "ls" or "objects", but for egress if it exceeds the storage 3 times. And Wasabi has a strange storage duration policy: https://wasabi.com/pricing/faq