gptscript-ai / gptscript

Build AI assistants that interact with your systems
https://gptscript.ai
Apache License 2.0
2.98k stars 261 forks source link

Better error handling when remote tool entry is invalid( had https:// for a remote github tool) #638

Open sangee2004 opened 2 months ago

sangee2004 commented 2 months ago

gptscript version - v0.0.0-dev-d3cac6e6-dirty

Steps to reproduce the problem:

  1. Execute the following script which has invalid remote tool entry - https:// for image generation tool entry
    
    % cat image.gpt 
    Tools: https://github.com/gptscript-ai/image-generation, sys.download

Generate an image of a a squirelle playing with acorn on a tree and return only the url of the image. Download the contents of the url to a file myimage with appropriate extension.


2. It fails with  context length exceeded error message since the entire html content of the tool directory gets passed as instruction.

gptscript --disable-cache image.gpt 09:33:39 started [main] 09:33:39 sent [main] content [1] content | Waiting for model response... content [1] content | imageGeneration -> {"defaultPromptParameter":"a squirrel playing with an acorn on a tree"} 09:33:41 started [https://github.com/gptscript-ai/image-generation(2)] [input={"defaultPromptParameter":"a squirrel playing with an acorn on a tree"}] 09:33:41 sent [https://github.com/gptscript-ai/image-generation(2)] content [2] content | Waiting for model response...09:33:43 usage [total=305] [prompt=280] [completion=25] 2024/07/16 09:33:43 error, status code: 400, message: This model's maximum context length is 128000 tokens. However, your messages resulted in 128642 tokens. Please reduce the length of the messages.



**Expected Behavior:**
There should be tool validation that would be able to catch this as an invalid tool entry and report it even before it get used for LLM calls. 
drpebcak commented 2 months ago

This is a tricky one because its not an invalid tool per se - its just too big.

You could serve a gptscript from a webserver that just said say hello and that would be valid syntax. Is there anything other than the length you would expect to trigger a validation error?