search

DevEmperor / WristAssist

A powerful ChatGPT and DALL-E app for all WearOS devices
https://play.google.com/store/apps/details?id=net.devemperor.wristassist
Apache License 2.0
93 stars 14 forks source link

Ability to change OpenAPI endpoint #13

Closed bfvogel closed 9 months ago

bfvogel commented 10 months ago

Hi I'm wondering if you might expose the ability to change the OpenAPI endpoint url so that users can integrate with other OpenApi compatible instances. My specific use case is a is an internal OpenApi compatible LLM that has ingested my companies private data. We currently use a web portal for general access but it would be nice to have access via our managed wearos devices for addhoc queries.

DevEmperor commented 10 months ago

Hey, yes, that's definitely possible. I already offer the option to switch to "pawan.krd". I will implement this as soon as possible so that you can also set customised hosts and models (!). However, it is important that the server responds exactly like OpenAI. Otherwise the API library in the background cannot interpret this correctly. However, you will have to analyse and test this, as I do not have access to your internal models. :)

DevEmperor commented 10 months ago

Hey, I've just added the feature. From the next update (v2.5.0), you can specify your own OpenAI-like API server in the settings. :)

I would appreciate feedback on whether this worked, as like I said I can't test it. :)

bfvogel commented 10 months ago

This is an amazing turn around BTW!. It seems to work perfectly. I tested it over HTTP and HTTPS with just the base url and things respond effectively.

bfvogel commented 10 months ago

One gotcha. I had to expose a public Internet endpoint for the API in order for WearOS to be able to reach it consistently. It look like the app (rightfully so) does not request wifi access which would be required for a purly internal network call.

Some sort of toggle and logic to perform

NetworkRequest.Builder() .addTransportType(NetworkCapabilities.TRANSPORT_WIFI);

based on if the Custom_url is RFC1918 would be a super clean way to do it.

bfvogel commented 10 months ago

Okay I got too excited. I'm going to do a bit of digging but the first thing I noticed is that with a my custom API centext is lost. My model accepts the same context as OpenAI gpt3.5 so I'm going to look into the client calls and see what differs. Also there is a "Stream" or max_tokens issue as responses are cut off. I'll try to get more real info. Thanks again.

DevEmperor commented 10 months ago

Thank you for your feedback. I will make sure that I add the feature that the app uses the WiFi connection to address private addresses as soon as possible. Regarding your last message, I would actually need more detailed information about what went wrong so that I can work with it. You can also send me your UserID (About -> long press on the WristAssist logo), then I can view stacktraces etc.