Open yberreby opened 1 year ago
And yes, I use
nativecomp
.
Do you use Emacs 29's builtin SQL library. I experienced a speedup after upgrading from Emacs 28 to Emacs 29.
@PRESFIL I tried Emacs 29 recently, but it breaks my Doom setup, and it actually slowed things down for me.
I believe org-roam uses emacsql, did you do anything special to switch to the native client?
did you do anything special to switch to the native client?
As far as I remember no. See help for org-roam-database-connector
variable.
For me it's value is 'sqlite-builtin
.
Thanks, I'll give it a shot once Doom has complete compatibility with 29.
This is tangential to my main point, though - (non-built-in) native modules & parallelism.
org-roam
is great, but slows down noticeably when working with many thousands of nodes. This is particularly unfortunate for very-frequently-used operations such asorg-roam-node-find
. I think it would be worth exploring the option of writing a native extension that internally uses multi-threading, at least for embarrassingly-parallel tasks such as mapping over all tasks.My own workflow involves writing a very large number of short
org-roam
nodes, with rich interlinking. I'm at 1700+ nodes after 7-8 months of daily usage. I love org-roam and I'd like to keep using it in this fashion, which is very effective for me, but the gradual slowdown is already felt and is unlikely to spontaneously get better. And yes, I usenativecomp
.In a more general fashion, I contend that performance is a feature, and that having the option to write parallel, native code and leverage external library ecosystems would be a great way to add features to org-roam. One particularly useful example would be semantic search (can be achieved with a small SBERT model such as
all-MiniLM-L6-v2
, or the more recentbge-small
), or efficient graph algorithms for discovering related notes that may not have direct links between them.Rust has a particularly good concurrency story, so it seems like a great choice for implementing native modules. ubolonton/emacs-module-rs enables writing Emacs native modules in Rust, and seems to have been used successfully by the elisp-tree-sitter project.
Maybe this would best be implemented as a pluggable, opt-in solution in case the use of a language other than Elisp detracts people from contributing; though in this case, there is the risk of feature divergence.
Even if no new features are implemented in the org-roam trunk, having a clear vision of how to cleanly integrate org-roam with native extensions would be valuable.
I am quite comfortable with Rust, though I am not very knowledgeable in Elisp at the time, nor with the internals of org-roam beyond the documentation & cursory looks at the code. Maybe I'm missing a piece of the puzzle regarding interoperability - happy to get some feedback.
Please check the following: