Closed CrackheadoooObsidianUserooo closed 10 months ago
sabaimran
commented
11 months ago What happens when you start Obsidian? Were you previously able to set it up and run it?
Khoj isn't able to find the file where your embeddings are supposed to be stored for your vault. This can happen perhaps if they weren't generated correctly the first time.
difhel
commented
11 months ago The same issue.
FileNotFoundError: [Errno 2] No such file or directory:
'/home/mf/.khoj/content/markdown/markdown.jsonl.gz'But I've never been able to get khoj to index the markdown files. This error is displayed from the first configuration.
difhel
commented
11 months ago I checked if there is the "markdown.jsonl.gz" file, but it seems that the folder "content" was not created yet, which is quite weird because I had configured files via "localhost/config" several times.
sabaimran
commented
11 months ago Ah, did you click on 'configure' in between? Is your config managed by obsidian?
I think this may be because the content is attempting to be read before it's configured. This would be a bug on our end, but let me see if I can reproduce it.
On Fri, Oct 6, 2023, 03:08 Mark Fomin @.***> wrote:
I checked if there is the "markdown.jsonl.gz" file, but it seems that the folder "content" was not created yet, which is quite weird because I had configured files via "localhost/config" several times. [image: image] https://user-images.githubusercontent.com/78644136/273175608-1c986f9b-155a-44d2-a7d4-2e007bf19a06.png
— Reply to this email directly, view it on GitHub https://github.com/khoj-ai/khoj/issues/488#issuecomment-1750335018, or unsubscribe https://github.com/notifications/unsubscribe-auth/APRMB23FB6S4Z5ZAZ3VGZ4DX57KA5AVCNFSM6AAAAAA5OJ6P3OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQGMZTKMBRHA . You are receiving this because you commented.Message ID: @.***>
lrq3000
commented
11 months ago Khoj isn't able to find the file where your embeddings are supposed to be stored for your vault. This can happen perhaps if they weren't generated correctly the first time.
This fixed the issue for me. The issue was that the first time around I had activated the offline chat feature, but it crashed with an OutOfMemory error when I clicked on the Configure button. The next time I restarted khoj, I got OP's error about a missing markdown index file. All I had to do was to let this buggy khoj continue running, access the control panel again, disable the offline chat model, and then click again on the Configure button, then the index file was correctly built.
I suggest khoj should try to do a check first about available memory, or at least manage more gracefully this exception to avoid crashing fully on the first time the user tries to Configure if they don't have enough RAM despite activating offline chat. Assuming OP's issue is the same as me.
/EDIT: I had to click on the Reinitialize button, otherwise with just the Configure button the interface would run but no pertinent result would be returned when doing a search. I can confirm that on m side, after doing Reinitialization, all features including search and offline chat work as espected.
sabaimran
commented
10 months ago Thanks for the detailing @lrq3000. I do think a lot of issues in particular occur when loading the offline chat model into application memory. It's a good idea to add an error/warning if we detected there's not enough available RAM to load it.
There may be a UX issue in the particular difference in the configure and reinitialize functionality as well. Generally, reinitialize is the best bet if there's any suspected misconfigured/corrupted index.
Checklist
Now I get this error when I Type Khoj
`