search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.88k stars 24.72k forks source link

Remove rollup functionality #112690

Open martijnvg opened 1 month ago

martijnvg commented 1 month ago

Rollup has been deprecated since version 8.11.0 (almost a year ago) en since version 8.15.0 it is no longer possible to create new jobs in cluster without rollup usage. We plan to remove rollup functionality as part of the 9.0 release.

elasticsearchmachine commented 1 month ago

Pinging @elastic/es-storage-engine (Team:StorageEngine)

thomasneirynck commented 1 month ago

@martijnvg

ES-Clusters in 8.x which have rollup indices, would they be able to upgrade to 9.0, or would this upgrade fail?

martijnvg commented 1 month ago

ES-Clusters in 8.x which have rollup indices, would they be able to upgrade to 9.0, or would this upgrade fail?

@thomasneirynck No, these clusters would need to migrate to downsampling before upgrading to 9.0.

lukeelmers commented 1 month ago

these clusters would need to migrate to downsampling before upgrading to 9.0.

@martijnvg Are you saying the existing rollup indices would be deleted, or just that the rollup jobs would fail to run but existing indices would still be accessible? Is ES logging a deprecation when it sees clusters with rollup jobs configured?

martijnvg commented 1 month ago

Are you saying the existing rollup indices would be deleted, or just that the rollup jobs would fail to run but existing indices would still be accessible?

I think we still need to determine this. If we chose to keep the ability to read rollup indices we need to at least keep some of the rollup functionality, but it could make migrating to 9.0 easier.

Is ES logging a deprecation when it sees clusters with rollup jobs configured?

Currently there non deprecation warnings in Elasticsearch. It is deprecated in the docs and from 8.15.0 clusters with no rollup usage can't define new rollup jobs (I believe rollup ui in kibana also was updated to show meaningful warnings/errors).

thomasneirynck commented 1 month ago

@martijnvg

wrt

If we chose to keep the ability to read rollup indices we need to at least keep some of the rollup functionality, but it could make migrating to 9.0 easier

do we know where we can track this discussion?