akamai / uls

Unified Log Streamer (ULS)
Apache License 2.0
28 stars 10 forks source link

[FEATURE] Aggregation formatting with concatenation (instead on JSON array) #45

Closed bitonio closed 11 months ago

bitonio commented 1 year ago

Is your feature request related to a problem? Please describe. Splunk HTTP Collector uses a variety of configurations to parse incoming payload containing more than one events. Some ULS admin may not have the full flexibility on the parsing.

Describe the solution you'd like We would need to be able to send a list of event concatenated with a configurable separator (space, newline etc...). The payload in this case would not be a strictly valid JSON structure.

Describe alternatives you've considered Keep older version of ULS

MikeSchiessl commented 1 year ago

Ok, let's be specific here. We're explicitly talking about the HTTP output, as this is the only one "buffering".

Currently ULS allows you to adjust the output format (ULS_HTTP_FORMAT) '{"event": %s}'. By default, ULS_HTTP_AGGREGATE is set to 500, which leads into replacing the %swith a json list as pointed out in the docs.

Current behavior Let's call that formattype: json-list Description: %s will be replaced by a list of logline entries Example: '{"event": [{logline1},{logline2},{logline3},{….},{logline500}]}'

Switchable behavior Let's call that formattype: single-event Description: Every event receives it's own wrapper (defined by format) but will not be concatenated Example: '{"event": {logline1}}{"event": {logline2}}{"event": {….}}{"event": {logline500}}'

Note: we need to re-consider introducing the line-breaking behavior after every event

MikeSchiessl commented 11 months ago

The feature was implemented in ULS v1.7.0 (released 2023-10-10)