search

hse-project / rfcs

HSE project request for comments (RFC)
https://hse-project.github.io
Apache License 2.0
1 stars 2 forks source link

Simplify internal CN stats #10

Closed alexttx closed 1 year ago

alexttx commented 2 years ago

Signed-off-by: Alex Tomlinson atomlinson@micron.com

beckerg commented 2 years ago

I think it'd be useful to have rollup stats to total bytes read/written by each job type so that we can easily see the distribution of media usage by job. Probably one per tree, with a rollup for the entire kvdb.

alexttx commented 2 years ago

I think it'd be useful to have rollup stats to total bytes read/written by each job type so that we can easily see the distribution of media usage by job. Probably one per tree, with a rollup for the entire kvdb.

Persisted? I'm a little leery about persisting it. When would it be reset?

I guess the title is a bit misleading, but this RFC is focused on kvset stats. I didn't name it "kvset stats" since they are rolled up to node and tree level.

beckerg commented 2 years ago

No, not persisted, just something we can dump via rest or at close. But if that's out of scope we can add it in whenever, we're really gonna need it when we have to starting figuring out what to throw out in the cloud environment (probably scatter remediation has to be disabled or set really high to let gc clean up scatter, or something....)

alexttx commented 1 year ago

Some worthy ideas here, but this would be a large code change and our focus is on other areas of HSE.