Open neptunian opened 4 years ago
Pinging @elastic/ingest-management (Team:Ingest Management)
There is a stack monitoring package that @ycombinator will be introducing in the future that will need ILM policies per dataset.
Strictly speaking it will be per package but we could define the same policy per dataset too, if that's all we decide to support.
@mostlyjason I don't think we need that with data tier? correct?
Yes we are planning a data tiers feature that will allow users to set retention and other settings. @mukeshelastic is helping to lead that
Currently all index templates point to the corresponding
type
of data ILM policy which comes installed in ES callslogs
ormetrics
. We need to support a custom ILM policies, either per package or per dataset or both. @ruflin recently made a change to a dataset manifest file to override index template mappings or settings: https://github.com/elastic/package-registry/issues/497. But I think the ILM policy file can be added per dataset in theilm_policy
directory before the index template is created and referenced there. The index template should reference the most specific policy. Most specific being the dataset and least the global). Policies will need to be tracked and removed like other installation assets.Note: There is a stack monitoring package that @ycombinator will be introducing in the future that will need ILM policies per
datasetpackage.