hjyssg / ShiguReader

硬核宅宅资源管理器. Ultimate Manga Resource Manager
MIT License
401 stars 45 forks source link

[启动性能优化] sqlite批量insert测试 #214

Closed hjyssg closed 1 year ago

hjyssg commented 1 year ago

有insert 22.225s scan complete. 22.164s scan complete.

注释掉,完全没insert 15.456s scan complete. 15.506s scan complete.

实测一秒10000条数据 三分之一的启动时间是sqlite的insert操作

hjyssg commented 1 year ago

https://www.sqlite.org/speed.html

hjyssg commented 1 year ago

https://www.cnblogs.com/hchxxzx/archive/2011/10/17/2215797.html https://blog.csdn.net/talkingmute/article/details/104089049

hjyssg commented 1 year ago

image

hjyssg commented 1 year ago

不知道从sqlite迁移到mysql,性能会提升多少

hjyssg commented 1 year ago

优化后的insertion表现

//时间秒/文件数
console.log("hdd without insertion")
console.log(1.139/1903*10000)
console.log(1.306/1903*10000)
console.log(1.184/1903*10000)

console.log("hdd")
console.log(1.184/1903*10000)
console.log(1.155/1903*10000)
console.log(1.124/1903*10000)

console.log("ssd")
console.log(1.586/2490*10000)
console.log(1.598/2490*10000)
console.log(1.525/2490*10000)

结论:scan的时候,固态和普通硬盘无差别 大约1000个文件,需要6.1秒

hjyssg commented 1 year ago

优化后的insertion表现

无insertion [chokidar] 1.699s scan complete. 2785 [chokidar] 1.847s scan complete. 2785 [chokidar] 1.769s scan complete. 2785

有insertion [chokidar] 2.161s scan complete. 2785 [chokidar] 2.064s scan complete. 2785 [chokidar] 2.149s scan complete. 2785

hjyssg commented 1 year ago

原来sort性能比_.sortBy强好多啊

https://www.measurethat.net/Benchmarks/Show/14335/0/sortby-vs-js-sort

hjyssg commented 1 year ago

参考batchInsert() 超级快

[batchInsert] file_table 52724 2068ms [batchInsert] tag_table 113603 1820ms