Feature Request: Support for Fetching Previous/Next Segments in VertexAISearchRetriever

[lgesuellip]

Hey Team,

I am working on this feature to support fetching previous/next segments in VertexAISearchRetriever.

I worked at Pampa Labs, and we discussed this feature on the Langchain Ambassadors slack channel. Erick Friis suggested tagging you @lkuligin.

Description

Currently, the VertexAISearchRetriever from langchain_google_community does not support fetching previous and next segments of documents for unstructured data. This is crucial for us to improve the quality of the fetched documents for our LLM.

The Google Discovery Engine API, including its Python library google.cloud.discoveryengine_v1beta, directly supports fetching previous and next segments. Relevant documentation can be found here.

Request

• Add support for fetching and mapping previous/next segments in VertexAISearchRetriever. Incorporate additional useful configurations from the Google API in the _create_search_request method.
• Compare the Google API’s capabilities (particularly for fetching segments and other configurations) with the current functionality in VertexAISearchRetriever and implement the handling of segments in the _convert_unstructured_search_response method.

Note

We are not addressing chunk support at this time, as it will be handled as a separate project.

[lkuligin]

SGTM

Would you like to send a PR, please?