Netflix / archaius

Library for configuration management API
Apache License 2.0
2.47k stars 486 forks source link

Property usage instrumentation initial framework #651

Closed akang31 closed 1 year ago

akang31 commented 1 year ago

Initial framework for capturing property usage in Archaius via an AccessMonitorUtil.

In this first iteration, AccessMonitorUtil keeps track of a map from PropertyId (new concept) to a wrapper around a List of property usage events (which only contain a timestamp). We are not overly concerned about the granularity of the property usage data at this stage, so we also only keep one such event per iteration (effectively just giving us a map from property id to last used timestamp). We leave the data model here generic enough to augment it with additional metadata at the high level, and property level (e.g. we could also add additional scope-related metadata, maintain a count of property accesses in the future, or keep track of more events per period).

AccessMonitorUtil also has a setDataFlushConsumer method, which when applied will on a (default) 2 minute cadence flush the stored property usage data to the method and reset the existing data. The idea is for this to be used to send to a remote database/stream for aggregation and analysis.

We initially only support instrumenting in the PollingDynamicConfig and propagate the settings through CompositeConfig. This is only because our primary target for instrumentation is remote properties. We should support instrumentation in other dependent configs (LayeredConfig, PrefixedViewConfig, and PrivateViewConfig) and there is nothing stopping us from supporting it in the other configs as well.

There are also changes in JsonPersistedV2Reader to hold onto property ids, and not just keys and values, so that we can uniquely tie usage data with an actual property when the data is passed along for analysis.

To enable AccessMonitorUtil monitoring in Guice, an AccessMonitorUtil must be bound along with the Persisted2ConfigProvider (this will also enable the JsonPersistedV2Reader reading property ids). To enable data flushing, the bound AccessMonitorUtil must have a DataFlushConsumer set.

In Spring, the idea will be the same, with the presence of an AccessMonitorUtil enabling monitoring, and a DataFlushConsumer being set enabling the flushing.

Followups:

rgallardo-netflix commented 1 year ago

Looks good. Have you tried making a local build and running it in an app?