NickSavage / Zettelgarden

A personal knowledge management system implemented with React and Go, inspired by the idea of the zettelkasten.
https://zettelgarden.com
4 stars 0 forks source link

Properly chunk long documents #4

Open NickSavage opened 2 days ago

NickSavage commented 2 days ago

Creating large cards currently chunks based on a set text size, then throws it all together. So a card may have 100 chunks, then that's summarized into only 10 relations (even though the system stores 1000 or so of them). This ends up obliterating things that aren't "hey this is also a podcast".

NickSavage commented 1 day ago

Probably want a separate chunks table, that will be recombined into the body of cards. This can be largely transparent to the frontend I think, except maybe jumping to other chunks.

Maybe make. asmaller chunk size as well, we'll have to see about that. per sentence? will add a lot of disk space.