Our customers frequently use client-side sampling to control costs, but we also don’t feel confident suggesting a 100% client-side sampling rate on all platforms because of performance concerns. At the same time, we tend to assume a 100% client-side sampling rate when we talk about the completeness of metrics.
Accurate metrics must be a key goal regardless of client-side or server-side sampling rates.
User Stories
As a user, I want my metrics to be accurate, even if I sample on the client.
Possible Solutions
Improve our SDKs to sample 100% in all cases (does not solve cost control by customers).
Always extract and aggregate complete metrics in our SDK regardless of the sampling rate (will create cost on our end, complicated to maintain on all SDKs)
Report the client-side sample rate with metrics sent to the backend and extrapolate them based on this rate.
Clarifications
While it's a technical requirement to strive for, we will never be able to assume that any given client samples with 100% and as such, tuning our SDKs for this, won't solve the problem completely.
Whichever approach we choose, this will increase COGS for us.
We need to re-validate all SDKs and implement better backpressure management to guarantee even if customers set 100% sampling, especially in a very high throughput environment, we never negatively impact the customers infrastructure
Problem
Our customers frequently use client-side sampling to control costs, but we also don’t feel confident suggesting a 100% client-side sampling rate on all platforms because of performance concerns. At the same time, we tend to assume a 100% client-side sampling rate when we talk about the completeness of metrics.
Accurate metrics must be a key goal regardless of client-side or server-side sampling rates.
User Stories
Possible Solutions
Clarifications