perf(processor/post): improve processing speed when `config.post_asset_folder` is enabled

yoshinorin commented 7 months ago

refs:
- https://github.com/hexojs/hexo/issues/2579#issuecomment-2053889785
- https://github.com/hexojs/hexo/issues/5456

I think other issues also related this PR

When slow?

config.post_asset_folder is enabled.

Why?

When config.post_asset_folder is enabled, Hexo calls the Post.toArray() function for each file. The time complexity of Post.toArray() is O(n). Additionally, Hexo uses the Post.find method to search for the post, and the average time complexity of find is also O(n).

So, if there are 100 articles and 100 files, the time complexity would be O(100 * 100 * 100) = O(n^3).

How to solve?

Avoid calling Post.toArray(). Instead, we will search based on whether the directory where the files are stored (assetDir) matches. Also, if the file found with PostAsset, use Post.findById. The time complexity of Post.findById is O(1).

Benchmark Environment

Below is bench env.

### Machine ``` # OS Microsoft Windows [Version 10.0.22631.3447] # Cpu AMD Ryzen 7 PRO 4750G with Radeon Graphics # Memory Capacity Name Tag 17179869184 Physical Memory Physical Memory 1 17179869184 Physical Memory Physical Memory 3 ``` ### Hexo and Node.js ``` $ hexo -v hexo: 7.2.0 hexo-cli: 4.3.1 os: win32 10.0.22631 node: 20.11.1 ... v8: 11.3.244.8-node.17 ``` ### `package.json` ``` "dependencies": { "0x": "^5.7.0", "hexo": "file://C:\\Users\\\\development\\hexo\\hexo", "hexo-filter-nofollow": "2.0.2", "hexo-generator-archive": "git+https://github.com/hexojs/hexo-generator-archive.git#master", "hexo-generator-category": "git+https://github.com/hexojs/hexo-generator-category.git#master", "hexo-generator-index": "git+https://github.com/hexojs/hexo-generator-index.git#master", "hexo-generator-sitemap": "git+https://github.com/yoshinorin/_hexo-generator-sitemap.git#master", "hexo-generator-tag": "git+https://github.com/hexojs/hexo-generator-tag.git#master", "hexo-pagination": "git+https://github.com/yoshinorin/hexo-pagination.git#my-site", "hexo-renderer-ejs": "git+https://github.com/hexojs/hexo-renderer-ejs.git#master", "hexo-renderer-markdown-it": "git+https://github.com/hexojs/hexo-renderer-markdown-it#master", "hexo-server": "git+https://github.com/hexojs/hexo-server.git#master" }, ``` ### `_config.yml` ```yml # Site title: Dummy subtitle: description: author: yoshinorin language: ja_JP timezone: Asia/Tokyo meta_generator: false # URL url: https://example.net root: / permalink: :year/:month/:day/:title/ permalink_defaults: pretty_urls: trailing_index: true # https://example.com/foo trailing_html: false # https://example.com/foo/index.html # Directory source_dir: source public_dir: public tag_dir: tags archive_dir: archives category_dir: categories code_dir: downloads/code i18n_dir: :lang exclude: - '**/node_modules/**' - '**/.git/**' skip_render: - '**/node_modules/**' - '**/.git/**' ignore: - '**/node_modules/**' - '**/.git/**' # Writing new_post_name: :year/:month/:day/:title.md default_layout: post titlecase: false filename_case: 0 external_link: enable: true field: site exclude: render_drafts: false post_asset_folder: true relative_link: false future: true highlight: enable: true line_number: true auto_detect: false tab_replace: # Category & Tag default_category: uncategorized category_map: tag_map: # Date / Time format date_format: YYYY-MM-DD time_format: HH:mm:ss updated_option: mtime # Pagination ## Set per_page to 0 to disable pagination per_page: 10 pagination_dir: page # Theme theme: tranquilpeak # I'm using a theme that I've delete many features from https://github.com/LouisBarranqueiro/hexo-theme-tranquilpeak. sitemap: path: "sitemap.xml" tags: false categories: false markdown: render: html: true xhtmlOut: false breaks: true linkify: true typographer: true quotes: '“”‘’' plugins: - markdown-it-footnote anchors: level: 1 collisionSuffix: 'v' permalink: true permalinkClass: header-anchor permalinkSymbol: ' ' nofollow: enable: true field: site ``` ### Theme I'm using a theme that I've delete many features from https://github.com/LouisBarranqueiro/hexo-theme-tranquilpeak.

Benchmark Time (Before, After)

Processed data

Number of posts: 1773
Number of post assets: 1784
Avg of post content length: 3645

Number of pages: 23
Number of page assets: 81
Avg of page content length: 4217

Number of tags: 246
Number of categories: 170
Number of routes: 5335

Time

Before	After
Cold Processing	33 sec	32 sec
Hog Processing	222 sec	23 sec

Command & Details

#### Cold Processing (run hexo clean before generate) ``` // run hexo clean before generate $ hexo clean $ Measure-Command { hexo g | Out-Default } INFO Files loaded in 19 s ... INFO 5335 files generated in 13 s TotalSeconds : 33.5191346 ``` #### Hot Processing (run hexo clean before generate) ``` // without hexo clean $ Measure-Command { hexo g | Out-Default } INFO Files loaded in 3.52 min ... INFO 1705 files generated in 8.64 s TotalSeconds : 222.3847807 ``` ## After #### Cold Processing (run hexo clean before generate) ``` // run hexo clean before generate $ hexo clean $ Measure-Command { hexo g | Out-Default } INFO Files loaded in 18 s ... INFO 5335 files generated in 12 s TotalSeconds : 32.9146049 ``` #### Hot Processing (run hexo clean before generate) ``` // without hexo clean $ Measure-Command { hexo g | Out-Default } INFO Files loaded in 12 s ... INFO 1705 files generated in 8.84 s TotalSeconds : 23.8749185 ```