Once Tiered Storage is deployed, it will be important to understand how remote data is accessed, and which patterns emerge on each deployment.
To do this, tiered storage logs should provide enough context about which client is fetching which partition/offset range and when.
At the moment, Client metadata is not propagated to the tiered-storage framework. To fix this, RemoteStorageFetchInfo can be extended with Optional[ClientMetadata] available on FetchParams, and have this bits of data available to improve the logging when fetching.
jeqo
commented
1 year ago
Once Tiered Storage is deployed, it will be important to understand how remote data is accessed, and which patterns emerge on each deployment.
To do this, tiered storage logs should provide enough context about which client is fetching which partition/offset range and when.
At the moment, Client metadata is not propagated to the tiered-storage framework. To fix this,
RemoteStorageFetchInfocan be extended withOptional[ClientMetadata]available onFetchParams, and have this bits of data available to improve the logging when fetching.