add Format: JSON Stream support

[tqangxl]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Windows event log to VictoriaLogs winlogbeat+Vector+VictoriaLogs

Vector：config [sources.source_winlog_event] type = "exec" command = ["D:\Lib\Logs\1.winlogbeat-8.10.4-windows-x86_64\winlogbeat-8.10.4-windows-x86_64\winlogbeat.exe", "-c", "D:\Lib\Logs\1.winlogbeat-8.10.4-windows-x86_64\winlogbeat-8.10.4-windows-x86_64\winlogbeat-My.yml"] mode = "streaming"

but： https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/#log-collectors-and-data-ingestion-formats

Log collectors and data ingestion formats

Here is the list of log collectors and their ingestion formats supported by VictoriaLogs:

How to setup the collector | Format: Elasticsearch | Format: JSON Stream | Format: Loki -- | -- | -- | -- Filebeat | Yes | No | No Fluentbit | No | Yes | Yes Logstash | Yes | No | No Vector | Yes | No | Yes Promtail | No | No | Yes

Log collectors and data ingestion formats Here is the list of log collectors and their ingestion formats supported by VictoriaLogs: How to setup the collector Format: Elasticsearch Format: JSON Stream Format: Loki [Filebeat](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Filebeat.html) [Yes](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html) No No [Fluentbit](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Fluentbit.html) No [Yes](https://docs.fluentbit.io/manual/pipeline/outputs/http) [Yes](https://docs.fluentbit.io/manual/pipeline/outputs/loki) [Logstash](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Logstash.html) [Yes](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html) No No [Vector](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Vector.html) [Yes](https://vector.dev/docs/reference/configuration/sinks/elasticsearch/) No [Yes](https://vector.dev/docs/reference/configuration/sinks/loki/) [Promtail](https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html) No No [Yes](https://grafana.com/docs/loki/latest/clients/promtail/configuration/#clients) ### Attempted Solutions _No response_ ### Proposal _No response_ ### References _No response_ ### Version _No response_

[neuronull]

Hello,

Are you using the http sink?

I see from the linked spec

VictoriaLogs accepts JSON line stream aka ndjson at http://localhost:9428/insert/jsonline endpoint.

Have you tried using the http sink with the json encoder, (https://vector.dev/docs/reference/configuration/sinks/http/#encoding.codec)

and the newline_delimited framing method? (https://vector.dev/docs/reference/configuration/sinks/http/#framing.method)

[hagen1778]

Thanks @neuronull @tqangxl !

I can confirm the following config works. Added example for configuring Vector to use VictoriaMetrics Json stream API here.