This PR adds ML processor as options in the context of search request and search response. Additionally, it finishes all of the logic in the input transform / output transform modals to dynamically fetch input / generate output. There are a few places where this preview / advanced transform view is gated:
Chaining multiple search request processors - in this case, since we don't have per-processor logging available, we cannot generate the interim transformed request. This is already tracked here: https://github.com/opensearch-project/OpenSearch/issues/14745
Advanced output transform for search request processors - we cannot do partial simulation of the search request processor in the context of output transformations, since there is no way to configure a working, simulate-able search request processor with an empty output_map - it is always required, as both input_map and output_map are optional in the context of ingest and search response. We need an empty output_map in order to isolate the response up to the point of configuring it.
More implementation details:
finishes handling the different contexts in InputTransformModal and the logic needed to fetch inputs
finishes handling the different contexts in OutputTransformModal and the logic needed to fetch inputs
refactors formikToIngestPipeline -> formikToPartialPipeline to be generic and handle for all contexts
adds gating in MLProcessorInputs to enable/disable the Preview button based on the different scenarios mentioned above ^
extra limitations on search request processor contexts: the input maps and output maps are required: warning modal added when these are empty
minor change in formatting of search responses to just show the nested _source document info. This is also to keep consistent with how the ML inference processors handle this input JSON, so we can replicate it correctly & consistently in the transform modals.
Testing:
Both added ML processors compile and work as expected, and generate valid search template resources
Preview works for both
Preview is disabled based on 2 scenarios above
Existing ingest processor & preview works as expected
Demo video, showing the 2 new processors, and preview functionality for both.
Resolves #217
Makes progress on #23
Makes progress on #219
Check List
[x] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Description
This PR adds ML processor as options in the context of search request and search response. Additionally, it finishes all of the logic in the input transform / output transform modals to dynamically fetch input / generate output. There are a few places where this preview / advanced transform view is gated:
More implementation details:
InputTransformModaland the logic needed to fetch inputsOutputTransformModaland the logic needed to fetch inputsformikToIngestPipeline->formikToPartialPipelineto be generic and handle for all contextsMLProcessorInputsto enable/disable thePreviewbutton based on the different scenarios mentioned above ^_sourcedocument info. This is also to keep consistent with how the ML inference processors handle this input JSON, so we can replicate it correctly & consistently in the transform modals.Testing:
Demo video, showing the 2 new processors, and preview functionality for both.
screen-capture (17).webm
Issues Resolved
Resolves #217 Makes progress on #23 Makes progress on #219
Check List
--signoffBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.