Heroic histogram data is currently computed locally.
It is practically impossible to aggregate percentile.
We are adding distribution to heroic to address that issue. Applications downstream will create data sketches that can be merged to compute percentile on the entire data distribution.
To support this approach, we changed the format of the metric.
Heroic consumers should be able to handle both the old and the new metrics.
We are creating a new version of Spotify100 serializer. This task will implement the logic to handle spotify100 new version metrics.
Reference
Current Format
Spotify100 version SCHEMA_VERSION = "1.1.0"
@Data
public static class Spotify100Metric {
private final String version = SCHEMA_VERSION;
private final String key;
private final Long time;
private final Map<String, String> attributes;
private final Map<String, String> resource;
private final double value;
}
Expected Format
Spotify100 version SCHEMA_VERSION = "1.1.2" (actual TBD)
@Data
public static class Spotify100Metric {
private final String version = SCHEMA_VERSION;
private final String key;
private final Long time;
private final Map<String, String> attributes;
private final Map<String, String> resource;
private final Value value;
}
Heroic histogram data is currently computed locally. It is practically impossible to aggregate percentile. We are adding distribution to heroic to address that issue. Applications downstream will create data sketches that can be merged to compute percentile on the entire data distribution. To support this approach, we changed the format of the metric. Heroic consumers should be able to handle both the old and the new metrics.
We are creating a new version of Spotify100 serializer. This task will implement the logic to handle spotify100 new version metrics. Reference Current Format Spotify100 version SCHEMA_VERSION = "1.1.0" @Data public static class Spotify100Metric { private final String version = SCHEMA_VERSION; private final String key; private final Long time; private final Map<String, String> attributes; private final Map<String, String> resource; private final double value; } Expected Format Spotify100 version SCHEMA_VERSION = "1.1.2" (actual TBD) @Data public static class Spotify100Metric { private final String version = SCHEMA_VERSION; private final String key; private final Long time; private final Map<String, String> attributes; private final Map<String, String> resource; private final Value value; }
Code Pointer: https://github.com/spotify/heroic/blob/5af661539d8a2f546433f95556aa5724bda8bf73/heroic-core/src/main/java/com/spotify/heroic/consumer/schemas/Spotify100.java#L137