search

brianpetro / smart-connect

82 stars 9 forks source link

Smart Connect (v1.1.53) and older shuts down when opening obsidian #9

Open ps003 opened 5 months ago

ps003 commented 5 months ago

Hello

I believe I'm using the latest versions of everything, but Smart Connect constantly shuts down. It doesn't matter if Smart Connect is open before I open/start obsidian. If I say to connect to smart connect inside of the plugin settings Smart Connect app closes with an error. If I have SC open before starting obsidian, Smart Connect crashes.

Macbook Pro: 16", 2021 CPU: Apple M1 Pro Memory: 32GB

Mac OSX: Sonoma 14.5 Obsidian: 1.5.12 Smart Connections Plugin: 2.2.6 Smart Connect: 1.1.53

let ms know what else you need. thanks!

brianpetro commented 5 months ago

Thanks for reporting this @ps003

Which embedding model are you using in the Obsidian plugin? 🌴

brianpetro commented 5 months ago

@ps003 I've been testing the local embeddings today, and I'm certain the cause to the issue is an embedding model that the computer cannot handle. I've been trying to "catch" the error so it can be handled better, like notifying the user to select a different model, but unfortunately, when this happens, Smart Connect simply shuts down as you described, and catching it in the process seems to make the software non-responsive. I'll be continuing to try to improve this, but that's the current status 🌴

ps003 commented 5 months ago

sorry it took me so long to reply, Brian. I have time and then it vanishes :)

I was using the Jina-v2-small-en (Local, 8,192 tokens, 512 dim) model.

I was using a rather large embedding model b/c I was curious if the dimension property and/or context length would give an advantage to the summary questions and more I was asking the system to do. I'm approaching this from a Private AI premise, so I'm trying to see what is capable when running on a local machine vs. online. If you have any guidance, please let me know. it sounds like I need to change to a different model, but not sure which one. I'll experiment... then I'd love to understand the implications of using smaller context windows and dimensionality. :)

brianpetro commented 5 months ago

@ps003 switch to BGE-micro-v2 and make sure "Block-level" embedding is also set to BGE-micro-v2.

Block-level embedding is an alternative to large context windows because it breaks notes into smaller chunks that fit into the smaller window 🌴

ps003 commented 5 months ago

Thanks @brianpetro - that worked. Do the alternate embeddings engines not crash for all users? Was curious why they'd stay in there if it's a known issue.

brianpetro commented 5 months ago

@ps003 it's highly dependent on computer resources, and some people have computers that are >10x others, there's quite a wide range on the market.

I definitely need to do something to clarify this to prevent issues. The ambiguity does cause a lot of problems. I was recently (when I wrote https://github.com/brianpetro/smart-connect/issues/9#issuecomment-2155502049) trying to detect the errors in a way that would let me notify users of an incompatibility, but unfortunately the error is seemingly un-catchable.

I keep a close eye on the development of relevant internals (i.e. transformers.js). And many of these models should work better on lower end computers in the future, but it's been taking some time to get access to the necessary optimizations.

🌴

ps003 commented 5 months ago

thanks for the background info, Brian! There is much to learn in this arena, so it's great to have friends and a guide. :)