search

CoatiSoftware / Sourcetrail

Sourcetrail - free and open-source interactive source explorer
https://www.sourcetrail.com/
GNU General Public License v3.0
14.94k stars 1.41k forks source link

Slow graph loading #932

Open Manouchehri opened 4 years ago

Manouchehri commented 4 years ago

With large projects (>1GB srctrldb), the UI takes several seconds to switch to a new graph or view. After the graph is loaded, the UI is snappy again.

I haven't done any profiling yet, but is this SQLite failing to scale here for huge databases? (Related: #406)

Manouchehri commented 4 years ago

After loading a project with perf record --call-graph dwarf ~/bin/Sourcetrail/Sourcetrail.sh, it's pretty clear that SqliteIndexStorage.cpp is where the majority of time is spent on huge projects.

time sqlite3 -readonly /mnt/builds/chromium/chromium.srctrldb 'SELECT id FROM edge WHERE source_node_id == 1;'
52
74
75
76
77
78
79

real    0m26.471s
user    0m0.777s
sys 0m0.222s
mlangkabel commented 4 years ago

@Manouchehri, thanks for investigating. If you want to take a deeper look, maybe our profiling classes help.

Manouchehri commented 4 years ago

Oh sweet, thanks! Didn't notice that page before.

I'd be willing to sponsor someone (aside from my existing Patreon membership) if they can make a large enough dent into speeding up large projects like the the Linux kernel and Chromium. =)