search

c0sogi / LLMChat

A full-stack Webui implementation of Large Language model, such as ChatGPT or LLaMA.
MIT License
245 stars 40 forks source link

Implement private memory #13

Closed Torhamilton closed 1 year ago

Torhamilton commented 1 year ago

We should use /imped (notice the p) to embed text to a private in-browser db. This project makes it possible Vector Storage

c0sogi commented 1 year ago

First of all, the project you provided uses Typescript, which is not compatible with Dart at all, so I can't use it.

There are a few things to think about first. It is possible to store vector data on the local device and query it from the database via vector operations.

However, this has the disadvantage of making the app very heavy. Not only does it make the app take longer to load due to the large size of the app itself, but it can also slow down overall performance due to the local DB lookup and vector operations not being optimized for vector operations.

Is storing and computing vector data on the frontend worth the effort?

Torhamilton commented 1 year ago

All valid points.