BrunoBonacci / mulog

μ/log is a micro-logging library that logs events and data, not words!
https://cljdoc.org/d/com.brunobonacci/mulog/
Apache License 2.0
490 stars 48 forks source link

`:data-stream` shows up as index in Elasticsearch 8 #90

Closed handerpeder closed 2 years ago

handerpeder commented 2 years ago

Using this elasticsearch publisher config:

 {:type        :elasticsearch
  :data-stream "clj-planner"
  :url         "https://my-url"
  :http-opts   {:basic-auth ["user" "pwd"]}}

The stream shows up as an index: image

As the title says I'm using Elasticsearch v8 on elastic cloud. Not sure if that is something you're supporting or not.

BrunoBonacci commented 2 years ago

Hi @handerpeder,

have you set up the data-streams before pushing the data?

On the Elasticsearch website, there is a document explaining the process. https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-a-data-stream.html

I should probably clarify this in the documentation, I will add a note.

From a client perspective, there is no difference between indexing records on a data stream vs a normal index, with the only difference that in data-streams the name of the index stays the same and doesn't contain a formatted date (like: my-index-2022-06-21)

BrunoBonacci commented 2 years ago

You can also convert an index alias to a data stream.

BrunoBonacci commented 2 years ago

added doc note https://github.com/BrunoBonacci/mulog/commit/bdca82cf554a0a10de05449ffc8d54dabf8aee88