search

justyns / silverbullet-ai

Plug for SilverBullet to integrate LLM functionality
https://ai.silverbullet.md/
GNU Affero General Public License v3.0
29 stars 1 forks source link

Disable indexing on clients, index only on server #44

Closed justyns closed 3 months ago

justyns commented 3 months ago

On my phone and also on my laptop, both in offline/sync mode - indexing basically caused a ton of lag or didn't work at all. It also caused a bunch of conflicted files.

As a workaround for now, I'm only going to generate and index embeddings on the server.

justyns commented 3 months ago

@zefhemel I'd be curious if you have any input/suggestions on this. Even if indexing on the client didn't cause actual issues - it'd still be repeating the work on each offline client.

Do you think it'd be feasible to always generate indexes on the server and sync them to the clients, instead of each client generating its own index? I guess with the caveat that new non-synced notes wouldn't be searchable.

zefhemel commented 3 months ago

I've been trying to avoid to implement database synchronization to be honest. Synchronizing files reliably is hard enough as it is.

Until we get a useful LLM that can run in a web browser I think realistically you would always need to be server connected to use any AI feature. Even if we'd have a browser based LLM, redoing all that embedding generation work on every client will likely never be a good use of energy.

Therefore I'd opt to only perform the indexing on the server and not in the client, even in Sync mode.