This PR adds many changes to the I/O and appearance on the drag-and-drop workspace. Note the final inputs/outputs/component types etc. are still not finalized. The purpose for adding placeholder components and updating inputs/outputs now is to iteratively improve on and make the current state of the plugin more functional and user-friendly, and to help start gathering initial feedback. Adding the existing use cases helps provide some immediate utility and testing of the corresponding backend Flow Framework plugin.
Description
This PR has 2 main changes: 1/ Onboarding a basic hybrid search use case, and 2/ providing visible, readonly flows on the DnD workspace for better overall e2e understanding from user perspective. More details:
adds several UI drag-and-drop components for both hybrid search use case (results transformer), and for search flow visibility (several query-related blocks)
modifies several existing components to handle inputs for both ingest and search (e.g., all transformers can now accept a document to be used on ingest, or a query to be used on search)
adds interfaces and functionality to create search pipelines
integrates the new use case into the prototyping components- Ingestor and QueryExecutor
makes resource descriptions on resource list tab more robust by determining type based on step type vs. resource type - this is due to a backend limitation of both ingest and search pipelines falling under the same pipeline_id resource type
minor refactoring of helper fns to handle more use cases
enhances searchIndex() end-to-end flow to process an optional search_pipeline param to propagate to the cluster. this is used behind-the-scenes in the QueryExecutor when executing the searches, if the use case is of type HYBRID_SEARCH.
And some minor tweaks to the reactflow workspace:
makes components draggable again, but still readonly
dynamically rendering color based on if the component needs input or is just a placeholder
Demo video:
selecting new hybrid search option
view of the end-to-end flows for both ingest and search (note all search components are readonly currently as we hardcode the results transformer/search pipeline)
creating and provisioning all of the resources needed for hybrid search, including a search pipeline
executing ingest
executing search with a hybrid query template, and using the created search pipeline behind the scenes
Screenshots of search flows added for other use cases:
Semantic search:
Neural sparse search:
Issues Resolved
Makes progress on #23, #68
Check List
[x] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Context
This PR adds many changes to the I/O and appearance on the drag-and-drop workspace. Note the final inputs/outputs/component types etc. are still not finalized. The purpose for adding placeholder components and updating inputs/outputs now is to iteratively improve on and make the current state of the plugin more functional and user-friendly, and to help start gathering initial feedback. Adding the existing use cases helps provide some immediate utility and testing of the corresponding backend Flow Framework plugin.
Description
This PR has 2 main changes: 1/ Onboarding a basic hybrid search use case, and 2/ providing visible, readonly flows on the DnD workspace for better overall e2e understanding from user perspective. More details:
IngestorandQueryExecutorpipeline_idresource typesearchIndex()end-to-end flow to process an optionalsearch_pipelineparam to propagate to the cluster. this is used behind-the-scenes in theQueryExecutorwhen executing the searches, if the use case is of typeHYBRID_SEARCH.And some minor tweaks to the reactflow workspace:
Demo video:
screen-capture (28).webm
Screenshots of search flows added for other use cases: Semantic search:
Neural sparse search:
Issues Resolved
Makes progress on #23, #68
Check List
--signoffBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.