Deployment is taking too much time

Atif-Sheikh commented 2 months ago

Check List

[X] I have already read Docs page & Troubleshooting page.
[X] I have already searched existing issues and they are not help to me.
[X] I examined error or warning messages and it's difficult to solve.
[X] I am using the latest version of Hexo. (run hexo version to check)
[X] My Node.js is matched the required version.

Expected behavior

It should take less than 10 minutes

Actual behavior

Takes 40+ minutes for deployment and Blog generation.

How to reproduce?

hexo g

Is the problem still there under `Safe mode`?

Yes

Your Node.js & npm version

v18.19.0
10.2.3

Your Hexo and Plugin version

No response

Your `package.json`

No response

Your site's `_config.yml` (Optional)

No response

Others

No response

stevenjoezhang commented 2 months ago

Currently, we are aware that in some cases, Hexo generation can be very slow: https://github.com/hexojs/hexo/issues/2579 https://github.com/hexojs/hexo/issues/4309 To answer your question, we need more information, such as whether your blog has a lot of articles or images?

Atif-Sheikh commented 2 months ago

Currently, we are aware that in some cases, Hexo generation can be very slow: #2579 #4309 To answer your question, we need more information, such as whether your blog has a lot of articles or images?

@stevenjoezhang We have around 600+ articles and yes every article has images as well.

Atif-Sheikh commented 2 months ago

@stevenjoezhang Can we expect the fix from the Hexo ASAP as most of the users are struggling with this issue?

stevenjoezhang commented 2 months ago

Did you encounter this problem after upgrading your Hexo version? As far as I know, slowdowns in generation time are not common, so I can't pinpoint the exact cause of this issue, nor do we know how to optimize it. If possible, you can help us by conducting an experiment: run the command

npm i -g 0x
0x node_modules/.bin/hexo g

This command will generate an HTML file (flamegraph), and we can use this information to identify where the performance bottleneck is.

yoshinorin commented 2 months ago

@Atif-Sheikh As mentioned in comment #2579, almost certainly your environment will be speed improvement by deleting db.json before running hexo g or hexo s, as suggested if you're not using the id for persistence purposes.

Of course, these times will be influenced by machine specs, the theme you're using, and the plugins, but there's no doubt it speeds up the process.

There are no official Hexo plugins that use ids for persistence purposes, but we don't know about third-party plugins.

In fact, I've been deleting db.json before running hexo g and hexo s for quite some time, and it's quite effective. In my current environment, I have 1800 .md files and 1500 image files (jpg, png), and db.json is 37MB. Without deleting db.json, hexo g takes about 8 minutes to complete, but with it deleted, it finishes in less than a minute.

Atif-Sheikh commented 2 months ago

Did you encounter this problem after upgrading your Hexo version? As far as I know, slowdowns in generation time are not common, so I can't pinpoint the exact cause of this issue, nor do we know how to optimize it. If possible, you can help us by conducting an experiment: run the command
npm i -g 0x
0x node_modules/.bin/hexo g
This command will generate an HTML file (flamegraph), and we can use this information to identify where the performance bottleneck is.

@stevenjoezhang Here is the flamegraph file, Please take a look at it and yes I'm using the latest Hexo version.

stevenjoezhang commented 2 months ago

@Atif-Sheikh I found that the hexo-related-posts plugin accounts for nearly 30% of the processing time. You could try disabling this plugin first.

stevenjoezhang commented 2 months ago

Additionally, the hexo-asset-pipeline also occupies a significant amount of processing time. This plugin has actually been deprecated by the Hexo team since 2022: https://github.com/hexojs/hexo-asset-pipeline. If you need to compress HTML files, I recommend using the hexo-optimize plugin, which uses a Rust-based HTML compression library for better performance. For image compression, you may need to look for other tools as alternatives.

Atif-Sheikh commented 2 months ago

Additionally, the hexo-asset-pipeline also occupies a significant amount of processing time. This plugin has actually been deprecated by the Hexo team since 2022: https://github.com/hexojs/hexo-asset-pipeline. If you need to compress HTML files, I recommend using the hexo-optimize plugin, which uses a Rust-based HTML compression library for better performance. For image compression, you may need to look for other tools as alternatives.

@stevenjoezhang Thanks for spending time on this issue, I'll try the changes you suggested and will get back to you.

hexojs / hexo