cst/inv: Download inventory reports to disk

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

https://redpanda.com

9.12k stars 556 forks source link

cst/inv: Download inventory reports to disk #19860

Open abhijat opened 1 week ago

abhijat commented 1 week ago

Download inventory report to disk periodically:

compute path hashes
store hashes to disk
add utility to load hashes and perform lookup for paths

Backports Required

[x] none - not a bug fix
[ ] none - this is a backport
[ ] none - issue does not exist in previous branches
[ ] none - papercut/not impactful enough to backport
[ ] v24.1.x
[ ] v23.3.x
[ ] v23.2.x

Release Notes

none

dotnwat commented 2 days ago

@abhijat is this PR replaced by your other inventory PR?

abhijat commented 1 day ago

@abhijat is this PR replaced by your other inventory PR?

It partially is replaced, I split off the other PR when this one got too large. When that one gets merged I will rebase this and start working on the service here, which will make this PR much shorter, the first few commits are common.