spegel-org / spegel

Stateless cluster local OCI registry mirror.
MIT License
1.15k stars 59 forks source link

More explanation of metrics #529

Open olicla opened 3 months ago

olicla commented 3 months ago

Describe the problem to be solved

Looking for a deeper explanation of metrics,

specifically:

Proposed solution to the problem

No response

phillebaba commented 3 months ago

This should be documented but I can give a quick answer first.

The difference between spegel_advertised_images and spegel_advertised_keys is that the former is a counter of the amount of images that have been advertised while the latter is a counter for the amount of images and the amount of layers that have been advertised. An image consists of multiple layers of varying amounts which is why this is interesting to know.

The metric spegel_mirror_requests counts the inflight registry requests and if they are mirrored or not. This is useful to know to determine for example how many mirrored requests are failing or if requests that are being served from disk are failing.

olicla commented 3 months ago

Thank you much.

If this is indeed documented somewhere, please share, as I have been unable to find.

Maybe misunderstanding, but on its own, how can spegel_mirror_requestsbe used to determine requests being served from disk are failing?

olicla commented 3 months ago

Further, I think I understand what exactly it means for an image and key to be "advertised" but I don't really see any surrounding docs. This would be helpful to define more precisely, in my opinion.

phillebaba commented 3 months ago

You have the label cache=hit|miss on the metric spegel_mirror_requests which indicate if the request resolved locally or not. This way you can determine the hit rate of requests to your cache.

I am probably not going to spend time writing docs on the details of how Spegel works anytime soon. Mostly because it requires a lot of background in both how OCI artifacts are composed but also how Kademlia works and is used in Spegel. I just dont have time for that right now.

olicla commented 3 months ago

Fair enough, are you able to explain exactly the difference between the source=external|internal label on spegel_mirror_requests? thanks for your time

AhmedTremo commented 5 days ago

@phillebaba I had same question, what's does source=external|internal refer to?

phillebaba commented 3 days ago

The source refers to if the request is coming from the same node or a different node. Right now Spegel configures two mirrors. The first one is for the local instance of Spegel and the second one is for any other instance within the cluster. This is for situations where Spegel is not running on the node for a some reason.

Does this answer your question?

AhmedTremo commented 3 days ago

yes, thanks @phillebaba

AhmedTremo commented 1 day ago

I just want to note down here, for future reference that there is something wrong with internal/external definition because from experience i saw zero external hits even though the pulls where serviced from other nodes and not from the external registry.