search

gptscript-ai / gptscript

Build AI assistants that interact with your systems
https://gptscript.ai
Apache License 2.0
2.82k stars 251 forks source link

Allow specifying headers for sys.http builtins #602

Open johnrengelman opened 3 weeks ago

johnrengelman commented 3 weeks ago

Currently, you cannot specify custom headers using the sys.http builts in (sys.http.get). The only parameter for get is the URL (https://github.com/gptscript-ai/gptscript/blob/main/pkg/builtin/builtin.go#L564), while post allows specifying the content and content-type (https://github.com/gptscript-ai/gptscript/blob/main/pkg/builtin/builtin.go#L607-L610).

Adding support for customer headers would allow tools to use the built-in functions to communicate with API endpoints that require tokens or api keys as headers to be invoked. It would also benefit from being able to customize things such as User-Agent headers or trace ids for requests.

cloudnautique commented 2 weeks ago

Hi @johnrengelman, this request makes sense, but if feels like leaving it to the LLM to decide how to call the tool could be error prone. The LLM could just make stuff up like an API key or token. The other issue with tokens/api keys are those values are likely not things you want to pass through the LLM directly, and there would be that possibility. How would you envision specifying them in a tool? Is it just auth type headers that would cover the majority usecase?

The openapi integration with GPTScript is pretty robust from what we have seen, it allows specifying the auth headers, parameters, etc, and gives the LLM context with the descriptions and example fields. It also has the benefit of being a built-in capability. Have you had a chance to look at it all? Do the APIs you use have specs? If not, I'm curious how they are documented for consumers.

johnrengelman commented 2 weeks ago

Hey @cloudnautique! Forgive me if I'm misguided, I'm only just starting to get into this space and work with this type of tooling. The situation I'm in is that I'm working with a bunch of internal APIs some of which have questionable or non-existent openapi specs. Those that do, are typically behind an authentication portal as well so just dropping a URL isn't always an option. I'm working on a POC that will wrap gptscript so I do have the ability to ship openapi specs with my wrapper, but that's going to become a maintenance/update problem if I can't be pulling from the source.

What I was thinking is that I could provide the key/token as a credential and then write a tool that says to make an http request to some url and ALWAYS provide the Authorization header with the value "Bearer $MY_TOKEN" in the prompt. Isn't this the use case for credentials in the first place? That should prevent it from shipping the value to the LLM?

Separate note on openapi specs...the one time I tried using it I ran into issues with tool size....lots of operations and the spec didn't have a good pattern for operationId that let me use the pattern matches in the import to break it up. Also ran into the missing operationId problem which has since been fixed, so I need to come back to trying this.

cloudnautique commented 2 weeks ago

You are bringing up some great points, and helping us see where the pain is, appreciate the discussion.

It sounds like one feature we need is the ability to get tools / openapi specs from authenticated endpoints. That one makes sense, and the work around (downloading and shipping everything together) works for now.

The other is that Auth header control within the builtin HTTP tools that supports using the credential framework. That is what the cred framework is for, but currently it is not optional if it is defined so a toggle for the builtins would be needed. Support for a custom Auth header as well. X-API-TOKEN or something.

As it relates to the openapi spec, given the state of adoption and consistency, custom local instances might be the way to go. Dynamically loading might not be viable until things get better. We did just merge a PR to autogenerate operationIds based on the verb and resources, so hopefully that ends up helping in the cases the operationId is missing.

cjellick commented 1 week ago

I think this is a good idea, but we should have a design discussion and come to some agreement on it before starting work.