search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.6k stars 8.21k forks source link

[Ingest Manager] How to support yaml and json for Elasticsearch assets #76360

Open ruflin opened 4 years ago

ruflin commented 4 years ago

The packages installed by Ingest Manager sometimes contain assets in YAML format and sometimes in JSON format. A good example here are the pipelines. This issue is to discuss on how we can support both in a more generic way.

The easy solution here would be to only support YAML or JSON and not both. Unfortunately both have their use cases and I think to make it as easy as possible but at the same time also powerful to build packages, we need both. Let me take the ingest pipeline as an example. If you take an ingest pipeline from Elasticsearch and just want to put it into a package, a user will export it as JSON and copy it there. This is also the format most of our users are used to. But if you build a pipeline manually, collaborate on it across pull requests and leave comments inside, YAML is often the format to go. This is why often the more complex pipelines in packages are in YAML and some simple ones in JSON.

Currently in the code we have for each assets where we support both formats and if/else check. Instead it would be nice if we would find a more generic way that if we add asset foo to be supported for Elasticsearch, the person implementing it does not have to think about supporting both. This indirectly also means, someone building a package does not have to think through if this specific assets is supported as JSON or YAML.

elasticmachine commented 4 years ago

Pinging @elastic/ingest-management (Team:Ingest Management)