github-actions[bot]
commented
1 year ago Pinging code owners:
- pkg/stanza: @djaglowski
See Adding Labels via Comments if you do not have permissions to add labels yourself.
Closed bin-shi-mulesoft closed 7 months ago
github-actions[bot]
commented
1 year ago Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself.
djaglowski
commented
1 year ago I agree this is a problem. From what I can tell, the scope_name parser itself is working correctly, but we are not considering the scope when translating to plogs. We just assume a single scope here.
@bin-shi-mulesoft, thanks for reporting this. Let me know if you want to take the issue and I'll assign it to you.
bin-shi-mulesoft
commented
1 year ago @djaglowski Sure. I can work on this.
github-actions[bot]
commented
1 year ago This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself.
someshkoli
commented
7 months ago Facing the same issue, not sure if the root cause described above still valid or not (@djaglowski can ya confirm ?). If yes, @bin-shi-mulesoft should I go ahead and pick it up ?
djaglowski
commented
7 months ago @someshkoli, my assessment of the cause is unchanged. I appreciate the help and will reassign this to you.
The biggest challenge here will be to preserve performance scope is not present. Grouping by scope will potentially introduce some overhead but we should be able to optimize for the case where scope is unused. Performance benchmarks will be helpful here. You may need to evaluate if the existing benchmarks in the package are sufficient to demonstrate the difference.
someshkoli
commented
7 months ago @someshkoli, my assessment of the cause is unchanged. I appreciate the help and will reassign this to you.
The biggest challenge here will be to preserve performance scope is not present. Grouping by scope will potentially introduce some overhead but we should be able to optimize for the case where scope is unused. Performance benchmarks will be helpful here. You may need to evaluate if the existing benchmarks in the package are sufficient to demonstrate the difference.
@djaglowski have benchmark numbers with me,
On main branch: BenchmarkEmitterToConsumer: ~3.12
On my branch: BenchmarkEmitterToConsumer: ~3.26
on my branch: BenchmarkEmitterToConsumerScopeGroupping (with 2 host variation and 2 scope variation): ~4.2s
rough implementation
resourceIdx, ok := resourceHashToIdx[resourceID]
if !ok {
resourceHashToIdx[resourceID] = pLogs.ResourceLogs().Len()
rl = pLogs.ResourceLogs().AppendEmpty()
upsertToMap(e.Resource, rl.Resource().Attributes())
scopeIdxByResource[resourceID] = make(map[string]int)
scopeIdxByResource[resourceID][scopeName] = rl.ScopeLogs().Len()
sl = rl.ScopeLogs().AppendEmpty()
sl.Scope().SetName(scopeName)
} else {
rl = pLogs.ResourceLogs().At(resourceIdx)
scopesInThisResource, _ := scopeIdxByResource[resourceID]
scopeInResource, ok := scopesInThisResource[scopeName]
if !ok {
scopesInThisResource[scopeName] = rl.ScopeLogs().Len()
sl = rl.ScopeLogs().AppendEmpty()
sl.Scope().SetName(scopeName)
} else {
sl = pLogs.ResourceLogs().At(resourceIdx).ScopeLogs().At(scopeInResource)
}
}@someshkoli, thanks that seems reasonable.
someshkoli
commented
7 months ago This groups the records by records -> scopes
with resource R1, R2 and scope S1, S2 -> total max possible log combination -> 4 (L1, L2, L3, L4)
grouping
[R1 -> (S1 -> L1, S2 -> L2)]
[R2 -> (S1 -> L3, S2 -> L4)]
mvillafuertem
commented
1 month ago Hi @djaglowski, I want to use the observed time as a timestamp, is this possible? any hint?
- type: json_parser
timestamp:
parse_from: 'observedTimeUnixNano'
layout_type: epoch
layout: nsotelcol-contrib[29061]: error decoding 'parse_from': unrecognized prefix
Component(s)
pkg/stanza
What happened?
Description
The Scope Name Parsing doesn't add the scope name to the ScopeLog.
Steps to Reproduce
echo "com.example.Foo - some message" > /tmp/example.logotelcol-contrib --config otel.yamlActual Result
Note: The output is one line, and I format it in json here for pretty looking.
Expected Result
The name of scope is set to
com.example.Foo.Collector version
v0.79.0
Environment information
Environment
OS: macOS Version 13.4
OpenTelemetry Collector configuration
Log output
Additional context
parse_fromin the example Configuration on Scope Name Parsing page seem not right. The value should beattributes.scope_name_fieldrather thanbody.scope_name_field