Refactor/chat sequence - Githubissues

bitcoinsearch / chat-btc

A conversational AI based on technical bitcoin sources

https://chat.bitcoinsearch.xyz/

MIT License

16 stars 13 forks source link

Refactor/chat sequence #25

Closed Emmanuel-Develops closed 1 year ago

vercel[bot] commented 1 year ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
chat-btc	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 8, 2023 7:33pm

adamjonas commented 1 year ago

In a side by side comparison comparison I'm getting more answers from prod from this branch. How can we verify?

Emmanuel-Develops commented 1 year ago

In a side by side comparison comparison I'm getting more answers from prod from this branch. How can we verify?

Yeah I noticed it too. We keep hitting the max token limit from open ai and that returns a 403 error. So we return a fallback error message

adamjonas commented 1 year ago

You are saying we are hitting this limit just in the streaming branch?

adamjonas commented 1 year ago

Still not performing as well:

Emmanuel-Develops commented 1 year ago

@adamjonas after extensive debugging there are no differences between the final prompt text passed to chat on this branch and that on prod. Retrying the same prompt on prod locally failed to produce any results and prod on live didn't as well. I used the same base query for all tests: what are the most important functions in bitcoin core, and disabled caching and reading cached answers so I only get chat responses.

I'll chalk it up to chatgpt not being exactly reproducible.

This branch did give an answer in one of my tests but that was 1 in about 8 instances of the same query so that isn't really usable.

IMG_01: one time answer from this branch IMG_02: prod branch on local failing the same query IMG_03: prod on live failing a reworded query (if not reworded it uses the cached no answer found message)

cc: @Extheoisah @urvishp80

adamjonas commented 1 year ago

Payments aren't working

Edit: I waited for a while with the payment pending. My alby wallet doesn't show a payment (I attempted 4-5 times), but voltage shows 2. If it did go through, it took a really long time for the modal to clear.

Streaming is working great though.

adamjonas commented 1 year ago

This isn't a blocker to merge but i'd like to unify the payment options rather than have 2 tabs:

We should present 3 options and just drop the 6-hour one

Also no one knows that 168 hours is a week. We should just call it 7 days.

github-actions[bot] commented 3 weeks ago

:tada: This PR is included in version 1.0.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: