[Spike]: Create an AI Chatbot

[Miaplacidus]

The ask here is to use Vercel's Next.js chatbot template (find repo here) to deploy a chatbot created by @RhysCommonwealth that has been trained on data from the 1Inch community.

Assuming there's an API key associated with the chatbot mentioned above that we can just plug into the template, this shouldn't be too involved. Maybe 2-3 points of effort.

Please add any relevant details @RhysCommonwealth below. @jnaviask @ForestMars Tagging you for details on how this should be prioritized.

---

V1?: POC - Show that we've done Streaming Chat with a Prompt Message

• [ ] Implement #6005 for AI Chat and ways to access it
  • (across desktop and mobile, per lofi design) (on desktop -- a button in top nav probably -I.e Chat(s) button), on mobile (is a bottom navigation item)
• [ ] Client Side Component for handling (in a React Component -> https://sdk.vercel.ai/docs/api-reference/use-assistant)
• [ ] Streaming Chat
  • Server Side Wrapper for Streaming Node.js Route -> https://sdk.vercel.ai/docs/api-reference/stream-to-response#example-nodejs-http-server
• [ ] Backend Needs Discovery (implement Chat Message as Data Model, idk what the model would be + other tests), naive approach is not "bad", but worth getting engineering eyes on this (CRUD routes)
  • May not need to save chat log history server side, may be able to use ChatGPT (they become 3rd party), can we save everything here (are there downsides to doing so?)
  • What might we want to save (Chat Room History, hashes so we can retrieve specific chats), (what's the level of effort)
  • Private and Public Chats (once public cannot be shared)

Assistant Training

• [ ] Assistants API (train on specific community data, fetch or add RAG to create Assistants)
  • add some ref to the Community Model (hasAssistant), feature flagged??
  • training (needs tech planning) Needs Discovery -> Option 1. Train Once and just grab via function calls / cron job at start, question then how often to update, how to know if threads are already in the store Option 2. Figure out how we just use Function calling and allow the 1inch bot to search via our API to find specific files and stuff Option 3. Both or more complex solution
• [ ] Be Able to mention different Agents / Communities / Files (per figma design)
  • use_mentions for char
  • Pull from any other thing

---

[dillchen]

Will bump during refinement! also @Miaplacidus please take a look at this doc as well

Very keen to discuss AI in our product beyond the scope of this specific ticket

[CowMuon]

The main thing here is to get a solid estimation of LOE involved.

That being said, it does not seem like this will be scoped for Cycle 1. Also, unclear how to estimate LOE in advance of UI mockups, which is the main thing needed here.

At the very least we should have a quick discussion what we are thinking here. Is this 'just' a chat widget that floats over existing screens?

Finally JTN, the idea of every community being able enable a concierge to answer questions about that community's respective governance or even on-boarding is terrifically exciting! We just need to be extremely diligent to hit our targets for Cycle 1, before we take on any additional scope.

[dillchen]

I think a productive way to move forward and to address Forest's questions would be to have Ifu and Forest make some comments here -- including a review of the Loom that's been put together:

https://www.notion.so/buildcommon/Common-Copilot-Embedding-AI-in-our-Product-374e3fed349e4bf78dfe7f1b6dc5598f?pvs=4

[Miaplacidus]

@dillchen @ForestMars Great Loom, btw! One thing my initial estimate (roughly 3 points) didn't consider is adding a drawer component. I see that this card has been created and assigned to me (thanks, Dillon). I'll be a little more conservative since I haven't played with this before and say maybe 3 to 5.

The generation of content feature should follow from training the bot on a community's content. The ability to paste into the editor does worry me a bit because Quill wishes to DESTROY ALL HUMANS. I'll be optimistic, though, and assume it will make an exception for me just this one time and that shouldn't be too crazy. I'll pull other engineers on that bit as needed.

In regards to things like making transactions, that feels more involved. My understanding is that ChatGPT just generates text (I've tried asking it to do things other than that and it politely tells me to stop being a silly meat sack). I don't believe it can make API calls, but I'm happy to do more investigating around that. We may have to modify the chat to take in specific commands and send an API call out to perform them.

Making a thread based on your convo with ChatGPT sounds awesome. But, I feel like those kinds of threads should have some kind of visual indicator in the thread listing page. In addition, seems like an opportunity to make something interesting on the actual thread viewing page, i.e. it shouldn't look like every other thread view page.

Overall, this sounds pretty amazing. Pardon if you've mentioned this before, but what was the feedback from users who've tried out the chatbot?

[dillchen]

I don't think @RhysCommonwealth has shared to users the chatbot yet.

Also @Miaplacidus please see the linked POC that Marcin has already put together on #6005: https://github.com/hicommonwealth/commonwealth/issues/6005#issuecomment-1867652466,

Seems like we should work off of that POC likely

[dillchen]

@Miaplacidus , I think u should convert this to a bucket ticket style ticket. For Front end, Streaming Chat etc. It's non fully trivial to use the template in our UI. For at least the reason we don't use next.js. Their streaming chat implementation assumes that.

Additional things that you have mentioned (I.e training on existing community data is further scope as well)

TLDR each of those, a bullet point, and we can discuss from there

[dillchen]

Next Steps: (Ideally each step is scoped to one week each

1. Discovery on items that needs discovery (timeboxed to 3 hours total)
2. Point Estimations for each for V1 PoC and Assistant Training

---

Requires approval from @ForestMars and @jnaviask (either way) for moving any of the tickets out of backlog