elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.51k stars 8.06k forks source link

[Migrations] Enable deferred migrations #158247

Open dokmic opened 1 year ago

dokmic commented 1 year ago

As part of #153117, the deferred migrations flag was disabled to prevent the contributors from using the feature.

See https://github.com/elastic/kibana/pull/153117#discussion_r1185044074 for reasoning.

After resolving #152807, the deferred migrations can be safely enabled for public usage. This can be done by reverting the following commit: https://github.com/elastic/kibana/pull/153117/commits/a0983e562664664a758b0af8de48a4a42fbe4467.

elasticmachine commented 1 year ago

Pinging @elastic/kibana-core (Team:Core)

rudolf commented 3 months ago

Another thing we should do is to expose the deferred flag through the modelVersions API as any new migrations would have to be written there. We should probably add this flag to unsafe_transform, data_backfill and data_removal change types.

pgayvallet commented 3 months ago

Note that in practice, "deferred migrations" serve no purpose in serverless, given everything is already "deferred" as it's executed by the migration job.

So my question was: given this fact, do we thing the traditional offering is a reason enough to continue with deferred migrations?

rudolf commented 3 months ago

[summarising sync discussion] At the moment cases-comments embed lens visualizations, so when there's a new lens migration cases-comments also need to be migrated. To avoid significant downtime cases sets the deferred flag on these migrations. But if cases ever needed to add their own migration using model versions this would not be possible anymore.

More generally, we want to advertise deferred migrations more widely. A lot of migrations could safely be deferred in classic which would help reduce upgrade migration downtime.

We also expect the effort to expose this via model versions to be a small task.