search

opensearch-project / dashboards-search-relevance

Tools to help search relevance engineers and business users tune search results for their OpenSearch applications.
Apache License 2.0
18 stars 34 forks source link

[FEATURE] [Search Comparison Tool] Display Images #286

Open sejli opened 1 year ago

sejli commented 1 year ago

Building upon #195

Is your feature request related to a problem?

As described in the previous issue, the current search comparison tool only displays textual document source. In the case there is an image URL in the document, displaying the image within the rows of the search result would be useful for comparing indices containing images.

What solution would you like?

Simply put, we want images to be displayed in the search comparison tool if a valid URL exists in the document source. If not, a placeholder image will be displayed within the row. We want the user to be able to customize this experience, with options not limited to:

  1. Which field in the document source contains the image
  2. Whether a base URL should be appended to the value stored in the document field

What alternatives have you considered?

There are no other ways to search and display images in OpenSearch.

Do you have any additional context?

The ML playground currently has a working implementation of displaying image, without much customization. As shown in this example images are displayed within the search result rows, and may be clicked on to be enlarged.

image
dtaivpp commented 8 months ago

@macohen @sejli would it make sense to create a new field type within OpenSearch that is an "image" type that is always a URI? By formalizing it a bit to make it a proper type we could do some interesting things like extracting the extension to it has a meta image type. This could also enable users to have a better experience with ingestion pipelines as they wouldn't need to pre-process their images to base64 to get them embedded.

macohen commented 8 months ago

It could make sense. I think we would want to separate that as a dependency here and have an issue in core, though. Would this also have an impact on kNN ingestion/multi-modal search? What do you think, @msfroh?