Node Google Search run failed: Failed to invoke tool: 'snippet'. (Results with no "snippet" not accounted for)

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.8

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

In workflow, create a Google Search Node from the Built-in Tool section
Set result type to "text". Run a search query that returns a result that does not have a snippet (result preview). For example "What is the parent company of Perry Elliss Inc?"
Note the tool returns an error Failed to invoke tool: 'snippet'.

My assumption is that this is due to the SERP API response containing a result which is missing a snippet, as I can see from the Google UI.

Proposed solution is to parse the SERP API results explicitly, checking for the presence of a "snippet" key in each result dictionary and only attempting to collect the snippet if it exists. It seems the current method assumes a snippet for each result, which is correct most of the time but will throw this error in some cases.

✔️ Expected Behavior

The concatenated snippets and links from each result are returned. In the case of a result with no snippet, only the link is included in the output.

❌ Actual Behavior

An error is returned, assuming because the tool was expecting a snippet key in the result dictionary within "organic_results". And the rare scenario where a result has no snippet was not accounted for.

To address the issue with the Google Search Node in Dify version 0.6.8, you need to modify the _process_response method in the SerpAPI class to handle search results that do not contain a snippet. This will prevent the tool from failing when a result lacks a snippet. Here is the updated code for the _process_response method:

@staticmethod
def _process_response(res: dict, typ: str) -> str:
    """Process response from SerpAPI."""
    if "error" in res.keys():
        raise ValueError(f"Got error from SerpAPI: {res['error']}")

    if typ == "text":
        toret = ""
        if "answer_box" in res.keys() and type(res["answer_box"]) == list:
            res["answer_box"] = res["answer_box"][0] + "\n"
        if "answer_box" in res.keys() and "answer" in res["answer_box"].keys():
            toret += res["answer_box"]["answer"] + "\n"
        if "answer_box" in res.keys() and "snippet" in res["answer_box"].keys():
            toret += res["answer_box"]["snippet"] + "\n"
        if (
            "answer_box" in res.keys()
            and "snippet_highlighted_words" in res["answer_box"].keys()
        ):
            for item in res["answer_box"]["snippet_highlighted_words"]:
                toret += item + "\n"
        if (
            "sports_results" in res.keys()
            and "game_spotlight" in res["sports_results"].keys()
        ):
            toret += res["sports_results"]["game_spotlight"] + "\n"
        if (
            "shopping_results" in res.keys()
            and "title" in res["shopping_results"][0].keys()
        ):
            toret += res["shopping_results"][:3] + "\n"
        if (
            "knowledge_graph" in res.keys()
            and "description" in res["knowledge_graph"].keys()
        ):
            toret = res["knowledge_graph"]["description"] + "\n"
        if "organic_results" in res.keys():
            for item in res["organic_results"]:
                if "snippet" in item.keys():
                    toret += "content: " + item["snippet"] + "\n" + "link: " + item["link"] + "\n"
                else:
                    toret += "link: " + item["link"] + "\n"
        if (
            "images_results" in res.keys()
            and "thumbnail" in res["images_results"][0].keys()
        ):
            thumbnails = [item["thumbnail"] for item in res["images_results"][:10]]
            toret = thumbnails
        if toret == "":
            toret = "No good search result found"
    elif typ == "link":
        if "knowledge_graph" in res.keys() and "title" in res["knowledge_graph"].keys() \
                and "description_link" in res["knowledge_graph"].keys():
            toret = res["knowledge_graph"]["description_link"]
        elif "knowledge_graph" in res.keys() and "see_results_about" in res["knowledge_graph"].keys() \
            and len(res["knowledge_graph"]["see_results_about"]) > 0:
            see_result_about = res["knowledge_graph"]["see_results_about"]
            toret = ""
            for item in see_result_about:
                if "name" not in item.keys() or "link" not in item.keys():
                    continue
                toret += f"[{item['name']}]({item['link']})\n"
        elif "organic_results" in res.keys() and len(res["organic_results"]) > 0:
            organic_results = res["organic_results"]
            toret = ""
            for item in organic_results:
                if "title" not in item.keys() or "link" not in item.keys():
                    continue
                toret += f"[{item['title']}]({item['link']})\n"
        elif "related_questions" in res.keys() and len(res["related_questions"]) > 0:
            related_questions = res["related_questions"]
            toret = ""
            for item in related_questions:
                if "question" not in item.keys() or "link" not in item.keys():
                    continue
                toret += f"[{item['question']}]({item['link']})\n"
        elif "related_searches" in res.keys() and len(res["related_searches"]) > 0:
            related_searches = res["related_searches"]
            toret = ""
            for item in related_searches:
                if "query" not in item.keys() or "link" not in item.keys():
                    continue
                toret += f"[{item['query']}]({item['link']})\n"
        else:
            toret = "No good search result found"
    return toret

This modification ensures that the tool does not fail with the error "Failed to invoke tool: 'snippet'" when the search results do not contain a snippet. Instead, it will include only the link in the output if a snippet is missing.

#### Details

**Sources** - [Run failed: Node DuckDuckGo search run failed: Failed to invoke tool: local variable 'snippets' referenced before assignment](https://github.com/langgenius/dify/issues/3076) - [api/core/tools/provider/builtin/google/tools/google_search.py](https://github.com/langgenius/dify/blob/main/api/core/tools/provider/builtin/google/tools/google_search.py) - [api/core/tools/provider/builtin/google/tools/google_search.yaml](https://github.com/langgenius/dify/blob/main/api/core/tools/provider/builtin/google/tools/google_search.yaml) - [en/tutorials/quick-tool-integration.md](https://github.com/langgenius/dify-docs/blob/main/en/tutorials/quick-tool-integration.md) - [sdks/python-client/tests/test_client.py](https://github.com/langgenius/dify/blob/main/sdks/python-client/tests/test_client.py) - [dev/pytest/pytest_unit_tests.sh](https://github.com/langgenius/dify/blob/main/dev/pytest/pytest_unit_tests.sh) - [api/tests/integration_tests/utils/test_module_import_helper.py](https://github.com/langgenius/dify/blob/main/api/tests/integration_tests/utils/test_module_import_helper.py) - [web/models/log.ts](https://github.com/langgenius/dify/blob/main/web/models/log.ts)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

langgenius / dify