ErikBjare / gptme

Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web, vision.
https://gptme.org/docs/
MIT License
2.64k stars 180 forks source link

Trigger tool detection only on complete lines to help nested code block start detection #251

Closed jrmi closed 2 weeks ago

jrmi commented 2 weeks ago

TL;DR; This MR fixes the bug when you have a markdown code block nested in another tooluse code block in streaming mode.

I think it's related to this issue #111

To reproduce

The response of this prompt looks something like that:

    [Some introduction text]

    ```save foo.md
    [In file explanation]

    ```python
    [python code block] 
[In file conclusion]
```
[end of llm answer]


If you are using streaming mode, the tooluse detection will stop immediately after the three back quotes preceding the `python` code block type because the check is executed after every single char.

# Solution proposed

The idea of this MR is to wait for the end of a line (i.e. a `\n` char) to execute the tool check. That way, we know the next three back quotes aren't the end of the code block as they are followed by the code block type (`python` here) and the tool use detection algorithm can handle that.

It's not perfect as it still won't detect if there is no code block type but it should work in more situation waiting for a more robust solution. 

Also I think it's a good idea to wait for the end of the line to try to detect the tools. I don't think we have benefits to execute it on every single char unless I missed something.
ErikBjare commented 2 weeks ago

Very good fix/workaround/improvement!

Thanks a lot :)

ErikBjare commented 1 week ago

This was certainly an improvement!

I stumbled into it failing to write this though:

Sure, let's save that:

```save rag.py
    ...
    return ToolSpec(
        name="rag",
        desc="RAG (Retrieval-Augmented Generation) for context-aware assistance",
        instructions="""
        Use RAG to index and search project documentation.

        Commands:
        - index [paths...] - Index documents in specified paths
        - search <query> - Search indexed documents
        - status - Show index status
        """,
        examples="""
        > User: Index the current directory
        ```rag index```

        > User: Search for documentation about functions
        ```rag search function documentation```

        > User: Show index status
        ```rag status```
        """,
        block_types=["rag"],
        execute=execute_rag,
        available=True,
    )

tool = init_rag()
```

(...message keeps going, doesn't get interrupted to execute save tool)

I assume this is due to the presence of ``` in the file, although they are indented, so they really shouldn't.