search

buchgr / bazel-remote

A remote cache for Bazel
https://bazel.build
Apache License 2.0
595 stars 154 forks source link

Question: How is CurrSize in curl http://localhost:8080/status caculated? #636

Closed WSUFan closed 1 year ago

WSUFan commented 1 year ago

We setup the maximum size as 550G when using bazel-remote-cache. However, it is different from what du command reports. By using curl, we have CurrSize as 306993414144 (~300G) but the du reports 500G+. So it will not trigger purging (forever). Some context: The service is running on K8s. We setup a S3 proxy backend. The "bazel-remote-cache disk" cache is stored in NFS (not using local disk). See below, image

Any idea? 

{
 "CurrSize": 306993414144,
 "UncompressedSize": 1697866801152,
 "ReservedSize": 0,
 "MaxSize": 590558003200,
 "NumFiles": 1365603,
 "ServerTime": 1675811496,
 "GitCommit": "b596a8dd20ef64e8f13540d9208089b4f30ddce8",
 "NumGoroutines": 573
}
WSUFan commented 1 year ago

I found that our NFS has a different block size. Maybe this is a reason that the program generates some mismatch? Since bazel-remote-cache using 4KBytes as the block size. image

mostynb commented 1 year ago

The CurrSize estimate is the sum of all the stored + reserved (in-progress uploads) blobs each rounded up to the nearest 4Kb. It's only an estimate, because calculating the true value (eg with du) is too slow to run on every cache update operation.

Rounding up to 4Kb blocks was chosen because that seems to be the most common value for linux filesystems. It would be nice if we could query the cache dir filesystem at startup to chose a more accurate block size, but even that wouldn't be perfect (IIRC some filesystems don't have a fixed block size, and the estimate would still ignore other filesystem overheads).

WSUFan commented 1 year ago

The CurrSize estimate is the sum of all the stored + reserved (in-progress uploads) blobs each rounded up to the nearest 4Kb. It's only an estimate, because calculating the true value (eg with du) is too slow to run on every cache update operation.

Rounding up to 4Kb blocks was chosen because that seems to be the most common value for linux filesystems. It would be nice if we could query the cache dir filesystem at startup to chose a more accurate block size, but even that wouldn't be perfect (IIRC some filesystems don't have a fixed block size, and the estimate would still ignore other filesystem overheads).

Yeah, thanks for letting me know.