[ML] Transforms: Wizard should "alert" me when using a data view with @timestamp instead of event.ingested

[philippkahr]

Describe the feature:

As the title says. We highly recommend to use event.ingested, to not miss any documents. https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-checkpoints.html#sync-field-ingest-timestamp

Still when using a data view that uses @timestamp you can miss documents and not get any information about it. Would it be possible to leverage the create data view on the fly without saving that is now available in Lens and Discover and base the transform upon that?

Even better would be, I select logs, transform alerts me about @timestamp, I can click ignore, change to event.ingested and the transform creates a temp data view for event.ingested, without me noticing anything.

Of course, that would require some kind of logic for the transform to verify that the event.ingested field is populated.

Describe a specific use case for the feature: Ux improvements.

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[walterra]

Thanks for the feedback! I'd like to summarize the current state of the wizard as of 8.7, then maybe we can identify some concrete steps how to improve.

• For the first step of the wizard we consider a Data View's date field to be used for the date picker and the "Use full data" button. This date field is not customisable at the moment. At the moment this date picker selection is only applied to this step for previews, the date range will not be applied to the transform to be created. It is more like a safety measure so users don't accidentally query possibly months/terabytes of data without any date filter.
• In the part of the form where you define the transform config, all date fields are available, so for example you can still group by a date field that's not the one defined in the data view.
• In the second wizard step as part of the config for continuous mode you again can pick any available date field and you are not limited to the one defined in the data view. We also have a note there saying Select the date field that can be used to identify new documents.. This is the place where you might want to use event.ingested. I don't think we can do much more here, because we have to keep in mind the wizard has to work with any kind of data and we cannot make an assumption based on the naming of a field to auto-pick a certain field or alert the user.

So looking at the above, the UI currently doesn't block you from picking the right date field, it might just not be that obvious. One thing we could do to make this more clear is to add the information which time field the date picker is using because that's currently not mentioned anywhere. As part of that we could add information/guidance that other date fields are still available in the rest of the form.