Add support for non-LLM AI provider API keys

[humphd]

We currently have a way to add API keys for OpenAI compatible LLM providers. In #715 I added support for using https://jina.ai without an API key to convert PDFs to Markdown, see https://github.com/tarasglek/chatcraft.org/blob/main/src/lib/ai.ts#L552

They also allow you to include an API key, if you send longer documents or use it a lot. It would be cool if we could add a personal API key and then have this code use it.

There are lots of providers that do this stuff, e.g., https://mathpix.com/. We could do a better job of allowing users to specify a "document provider" in the settings with an API key and using that when we convert certain file types.

[Kannav02]

Hey,

I was looking into this issue and I found 2 possible ways to get this working, In the following image, it can be seen that the option to add providers can be facilitated by the user

First Method,

Maybe we can inform the user about the fact that they can add the API key for specific providers to enable premium features without getting rate limited,

As for the developer side, since the URL would be a constant here,we can uniquely identify different providers based on their URL, preferably using a Map Data structure here and storing the providers

Second Method,

I found for only specific use cases certain providers are being used, what we could do is if that certain provider is being called as a function of the prompt, before proceeding with the request we could provide a UI pop-up to the user asking if he wants to provide an API key, similar to how ChatGPT does with custom GPTs before facilitating the request and the API key could be persisted just for the duration of the chat maybe?

      } else if (file.type === "application/pdf") {
        const markdown = await pdfToMarkdown(file);
        onFile(file, markdown);

[humphd]

In the Settings dialog, let's separate "LLM Providers" from "Other AI Providers" so it's clear they aren't the same thing.

[Kannav02]

sure, that sounds great, just to follow up on this, do you have a specific design for the same that you want to follow, if not I can work on getting the design and this issue resolved

thank you!

[humphd]

Since we only have a single non-LLM provider for now, I'd keep this simple:

1. Change the text "Providers" to "LLM Providers"
2. Add a new section below "LLM Providers" for "Other AI Providers" which will look similar to the one now, but no need for API URL I don't think (e.g. table of name, api key)
3. Hard-code jina.ai as the first and only one we support

[Kannav02]

Sure that sounds great, can I work on this?

[humphd]

@Kannav02 sure, go for it.