brianpetro / obsidian-smart-connections

Chat with your notes & see links to related content with AI embeddings. Use local models or 100+ via APIs like Claude, Gemini, ChatGPT & Llama 3
https://smartconnections.app
GNU General Public License v3.0
2.78k stars 183 forks source link

Smart Connect App (1.3.7) stuck in 'Making Smart Connections... Embedding progress: 210 / 16803 25571 tokens/sec using TaylorAl/bge-micro-v2' #857

Open sjhogerzeil opened 2 weeks ago

sjhogerzeil commented 2 weeks ago

Obsidian and all plugins up-to-date Smart Connect App cannot finish the embedding. Reloading, re-installing Smart Connect app and Smart-Connections plugin -> no result

I have attached the errors from the console log of Smart Connect app.

Here are the settings:

{ "smart_blocks": { "embed_model": { "model_key": "TaylorAI/bge-micro-v2" }, "embed_blocks": true }, "smart_sources": { "embed_model": { "model_key": "TaylorAI/bge-micro-v2", "legacy_transformers": true, "TaylorAI/bge-micro-v2": { "gpu_batch_size": 10, "min_chars": 300 } }, "pdf_adapter": { "chat_model": { "openai": {} } }, "image_adapter": { "chat_model": { "openai": {} } }, "smart_change": { "active": true } }, "file_exclusions": "Untitled", "folder_exclusions": "smart-chats, readwise, journals, assets, export, kindle-highlights", "is_obsidian_vault": true, "excluded_headings": "" } 20241029-000244

20241029-000246

20241029-000245

20241029-000247

brianpetro commented 2 weeks ago

Thanks for the detailed bug report and screenshots 😊

I tried a few different ways to reproduce the issue, but I haven't had any luck.

Can you create a "test vault" that reliably reproduces the issue?

This way I can dig in to what might be happening 🌴

sjhogerzeil commented 2 weeks ago

I may have found the problem. It seems that Smart-Connect is unable to process filenames containing the '#' sign.

All my files get embedded, except those meeting this criterion.

I have attached a one-file test-vault that reproduces the error reliably I think.

test-vault.zip

brianpetro commented 2 weeks ago

Oh yeah, that's got to be it. # in file names is currently not compatible.

It should be better handled, though. As in, providing a warning and not breaking things.

I'll have to see what I can do.

Thanks for the test vault 🌴

brianpetro commented 2 weeks ago

@sjhogerzeil the error should be prevented in the Obsidian Smart Connections plugin as of v2.2.85/2.3.26

The patch will also be included with the next Smart Connect app update (>=1.3.10) 🌴

MrKyomoto commented 3 days ago

@sjhogerzeil Hi , I'd like to know how to get the plugin's setting's file? I incorrectly change the AI model and now I met the same problem as you had met, moreover, my obsidian doesn't work when the Embedding process started and stuck at 10/1421 tokens, now I want to change my AI model to see if I can solve the problem but before the embedding process done, I can't change the AI model through the plugin's settings from obsidian

MrKyomoto commented 3 days ago

image you see, I change the default AI model, and I tried to delete the plugin and reinstall it ,but the model is pointed to the one I changed before.... so I thought get the setting file might be a solution (cuz I can change the content of the file directly)

sjhogerzeil commented 3 days ago

You might try changing the (hidden) smart_env.json file in your main vault folder.

It’s inside the (hidden) .smart-env/multi/ directory.

On 11 Nov 2024, at 17:21, Kyomoto wrote:

@sjhogerzeil Hi , I'd like to know how to get the plugin's setting's file? I incorrectly change the AI model and now I met the same problem as you had met, moreover, my obsidian doesn't work when the Embedding process started and stuck at 10/1421 tokens, now I want to change my AI model to see if I can solve the problem but before the embedding process done, I can't change the AI model through the plugin's settings from obsidian

-- Reply to this email directly or view it on GitHub: https://github.com/brianpetro/obsidian-smart-connections/issues/857#issuecomment-2468558614 You are receiving this because you were mentioned.

Message ID: @.***>

MrKyomoto commented 3 days ago

That helps! Thanks a lot 🥰 now the embedding process is ongoing although remains a long time xd