Add Support for Perplexity AI

[nkgokul]

Add support for using https://www.perplexity.ai/ similar to how we can openai and azure models right now

[agu3rra]

Referencing Perplexity API docs here in case it is useful for adding support: https://docs.perplexity.ai/api-reference/chat-completions

[agu3rra]

Existing Obsidian Copilot feature

Comparing Perplexity docs with those of OpenAI's ChatGPT's API docs, it looks like it uses the same exact endpoint /chat/completions. I remember the Copilot Obsidian plugin has an area for Custom API's which adhere to the specs of OpenAI's. I will make some experiments with it using my Perplexity API key and report back here.

What makes Perplexity special

TLDR: references.

LLM's are known for making stuff up, hence one of the distinguishing features of Perplexity that make it really stand out is the fact that it references content it quotes by default. I've cases where the LLM would make false claims, which I was able to check quickly because I had a way to look at references.

A couple of the items I noticed missing in the API spec of OpenAI and that would likely bring the most value out of using Perplexity are:

• return_citations: bool (default: false) - Determines whether or not a request to an online model should return citations. (docs mention it's in closed beta).
• return_related_questions: Determines whether or not a request to an online model should return related questions. Also in closed beta.

So even if those are added as a special vendor flavor in Obsidian Copilot, whoever holds the key would have to be in that closed beta for it to work. PS: I didn't check if the API just happens to return that data anyway even if I am not aware of being part of a beta program.

[agu3rra]

Tests

On a Windows 11 PC with Obsidian Copilot already enabled: I have added one of the available models from Perplexity as a custom model in the plugin settings as follows:

Blocked by CORS policy

You have to ensure you mark the CORS checkbox, otherwise it will error like this:

Timeout

After enabling CORS, I see what looks like a timeout error. I tried the following and still got the same results:

1. generating a fresh API token at Perplexity (in case mine was messed up)
2. Using the base URL as https://api.perplexity.ai
3. Using the base URL as https://api.perplexity.ai/chat/completions

Testing with curl

I made the API call to Perplexity with curl and observe it works normally with my token.

[agu3rra]

I was unable to individually delete the custom models without resetting the plugin entirely, so I opened https://github.com/logancyang/obsidian-copilot/issues/599

[agu3rra]

@logancyang I re-ran the test after updating the plugin to version 2.6.2, but the apparent timeout remains the same.

[logancyang]

@agu3rra Perhaps it's because perplexity api doesn't conform with openai format since it has those extra fields? We don't see the actual errors so it's a guessing game.

I'm considering hosting a gateway for perplexity and make it available in Copilot as a paid service, so people can subscribe with me instead of having their own perplexity API keys. WDYT? I'm a bit worried about the usage though, if I go for a subscription I may get a huge bill when people abuse it.

[agu3rra]

Hello @logancyang!

on the error

Perhaps I can to proxy all my traffic to BURP and see if stuff coming out of Obsidian to Perplexity yields more meaningful errors. I cloned the repo here to see if I can do anything useful, but I've never developed any Obsidian plugin or any TypeScript for that matter.

on the gateway idea

I think your gateway could be a way to monetize stuff for sure. I am not sure how large of an audience Obsidian is, but there's certainly value in being able to just query AI in the context of one's Obsidian Vault. You just need to make sure any abusers are metered appropriately so you don't pay the bill on their behalf. Perhaps do it like OpenAI, where they charge you in advance and you can keep auto-charging your card to certain limits. If a person wants to abuse

I understand a gateway with added benefits is what Perplexity is if you really think about it. They make a decent UI and gear their product towards effective search. I also am a subscriber there since it functions as a gateway to other LLM's. As in, if you pay 20 USD to OpenAI you get GPT-4 usage, Claude for Anthropic on the same amount. But 20 in PPL, gives you both plus the effective search, plus easy access to open models like Llamma or the Flux.1 image generation thing.

premium features idea?

I actually like the fact that your plugin allows one to freely select any vendor they are already using. I think it contributes to the amount of users it's receiving. So perhaps you can add some sort of premium feature that would get people to pay extra? Not even adding your gateway would be required. Right now I am strongly considering acquiring a subscription of Infranodus. I like the idea and possible insights it brings and now they added an Obsidian plugin to make it more appealing to me: https://infranodus.com/obsidian-plugin.

[logancyang]

@agu3rra These are great points. And thanks for the pointer to infranodus, I've seen one of their youtube videos once and it's quite fascinating.

I've been thinking about premium features for a while and would like to do it by providing enough extra value on top of people's API costs.

IMO my (or most) Obsidian plugin faces this monetization challenge: it's open source and completely client-side. So anyone who's a little motivated can check the code and go around any paywall-guarded logic.

Although I subscribe to a local-first and privacy-focused ideal, considering Obsidian plugins don't have an official blessed way to monetize, the only best way to offer premium features is through a hosted backend service.

I agree that the amount of users this plugin has is largely a result of the coverage of different APIs and the customizability. But it comes at a cost. "Highly customizable" means "little control" from the creator. Too many things can go wrong when a user configures in their own way. For example, a high-quality Vault QA experience requires an opinionated setup but the user is usually not an expert in that.

I'm considering offering a "Copilot+" mode in addition to the current modes where I have a more "work out-of-the-box" setup so people can see the difference vs. their own setup.

I also thought about providing knowledge graph support like visualization and even GraphRAG, but the ROI of that is not yet convincing enough.

For now, the "premium features" I have in mind are mostly special "slash command" skills that are only available in Copilot+ mode. These skills will need to call my backend to function, but the notes and search index are still stored locally. If you are interested I can sign you up as an early tester and you can use it for free. You are welcome to suggest features too! Feel free to email me for any comments and suggestions logan@brevilabs.com

[agu3rra]

@logancyang Going back to the premium Copilot+ backend idea then...

Perhaps a nitched market are Obsidian users that are not that technical and want the Copilot plugin to work out of the box? You charge them pre-paid model to avoid any upfront costs on your end and you interface to a number of different LLM vendors + host open-source models. The setup asks the user for an API key, which they acquire on your website (like Infranodus) and they call your API instead one of the vendor ones. I wonder if you can get some sort of discount from the vendors (OpenAI, Anthropic, etc) to make a good profit margin. I was watching this Lex Fridman podcast interview with Pieter Levels and that was just the part he was mentioning on monetization challenges: he would just reach out to vendors and be surprised getting 20-30% discount without a lot of effort. If you bring massive amounts of usage to their API's they want to keep you around.

I think that's a use case I'd pay for too. I am considering getting out of Perplexity Pro, now that I am paying for usage on OpenAI via the API. If Obsidian Copilot+ offered me a similar pre-paid option with the benefits of:

1. Running seamless on Obsidian.
2. Option to select from different LLM's with the click of a button. I think it's a compelling use-case.

I wonder if there's an effective way to forecast how many users would want that. Another point in the forecast that Pieter makes is that it's really hard in his opinion convert free users into paying ones. Hence why he always launched products that asked for $$$ after a small demo/taste.

[logancyang]

PR merged. Perplexity API will work in the next release, don't forget to turn on CORS.