opensearch-project / dashboards-flow-framework

A UI designer for constructing AI applications with OpenSearch
Apache License 2.0
9 stars 6 forks source link

Add partial support of advanced input transformations (ingest) #220

Closed ohltyler closed 1 month ago

ohltyler commented 1 month ago

Description

This PR begins the process of supporting the ML inference processor form inputs to support an advanced transformation flow. The scope of this PR is to set up the base form changes to the input map / output map inputs, the base transformation modals for configuring advanced input / output flows, and partial support of input transformations in the context of ingest. More specifically:

Demo video, showing the advanced flow. Expected input to each processor is shown. For the first/initial processor, the expected input is simply the list of documents. For the second processor, the expected input is a transformed version of the document, containing the embeddings values. Internally, this creates an ingest pipeline containing all of the preceding processors and executes _simulate against the provided documents. _Note a single inputmap transform is made to convert the hello field to the model's expected input field.

demo

Additional note: there is no defined UX on this, subject to change. But core functionality will remain the same.

Issues Resolved

Makes progress on #23

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

ohltyler commented 1 month ago

mend failure can be ignored - it is occasionally failing to run and marking as failed. No new deps are added in this PR.