search

parseablehq / parseable

Open Source ElasticSearch Alternative. Parseable helps you search and get insights from your logs in the most simple way possible.
https://parseable.com
GNU Affero General Public License v3.0
1.85k stars 95 forks source link

Support systemd-journal-remote for log ingestion #904

Open udf2457 opened 2 weeks ago

udf2457 commented 2 weeks ago

Please consider supporting systemd-journal-remote for log ingestion.

Docs: https://systemd.io/JOURNAL_EXPORT_FORMATS/#journal-export-format https://www.freedesktop.org/software/systemd/man/systemd-journal-upload.service.html# https://www.freedesktop.org/software/systemd/man/systemd-journal-remote.service.html#

Example from parseable "competitor": https://sematext.com/docs/logagent/input-plugin-journald-upload/

The core benefit is that systemd-journal-remote is clearly available out-of-the-box on many Linux platforms, and it pushes over http/https to a host. No need for third-party agents.

nitisht commented 2 weeks ago

hey @udf2457 thank you for the issue. Since Parseable allows dynamic schema, you should be able to ingest data from HTTP(s) calls already. Not sure if there are server changes needed to achieve this.

Have you tried the ingestion? To quickly test this you can point the ingestion call to our demo server's ingestor: http://ec2-3-136-154-35.us-east-2.compute.amazonaws.com:443/ (admin, admin).

udf2457 commented 2 weeks ago

Hi @nitisht

According to your docs....

The Parseable server detects the schema from first event

So I guess the question is whether you are setup to detect Journal Export Format, which AFAIK is binary ? 😉

nitisht commented 2 weeks ago

So I guess the question is whether you are setup to detect Journal Export Format, which AFAIK is binary ? 😉

This is indeed a better question! 😄 Right now we don't support binary formats. Let me look deeper into the format