Creating a gif from ugoira makes CPU run excessively hot

[Pxartist]

Describe the bug A clear and concise description of what the bug is. Creating a gif from ugoira makes my CPU jump from 45 - 55 degrees Celsius to 85+ degrees Celsius. In comparison to using the other features.

Downloading zip file: 50 - 60 degrees Celsius Downloading apng: 65 - 75 degrees Celsius Downloading webm: 65 - 75 degrees Celsius

Downloading anything using pixiv toolkit takes around the same time. However, the temperature deviation of the gif option is certainly too much.

Running prime95 Torture Test Small FFTs (all cores/threads) for 1+ hours: 72 - 75 degrees Celsius Running CinebenchR23 multicore stability test (30 minutes): 70 degrees Celsius Running CinebenchR23 single core stability test (30 minutes): 72 degrees Celsius

CPU temperature in BIOS: 37 - 41 degrees Celsius CPU temperature idle on Windows 10 with non-essential services off: 41 - 45 degrees Celsius CPU temperature idle on Windows 10 with Chromium browser running in background: 45 - 51 degrees Celsius CPU temperature on Windows 10 while actively browsing on Chromium browser: 48 - 55 degrees Celsius

Expected behavior A clear and concise description of what you expected to happen. Creating a gif from ugoira should not be more intensive of that of benchmarking software, whose workloads deviate far from reality and serve as means to test the limits of the CPU. If creating a gif from an ugoira is that CPU intensive, pixiv-toolkit will do well as the next generation CPU thermal limit benchmark.

CPU temperature should be similar to that of downloading zip file, apng, and webm, so I would consider 65 - 75 degrees Celsius to be more reasonable.

Example work from Pixiv If applicable, add work url to help me to reproduce the problem.

Any ugoira will cause this issue. The longer and more complicated the ugoira is, the longer the CPU will remain at the elevated temperature.

Screenshots If applicable, add screenshots to help explain your problem.

Browser and Version (please complete the following information):

• e.g. chrome, edge，firefox

Brave Browser v 1.26.77

Settings of extension You can export the settings of extension in setting page.

(JSON file type upload not supported on github) Ugoira settings: Rename ugoira file: null{id} Quality: Best Pack Frames Info: Type 2 Extend duration: off Save ugoira in relative location: not set

Additional context Add any other context about the problem here.

Downloading a gif from ugoira on my laptop does not result in such excessive temperatures (68 - 77 degrees Celsius; 45 - 55 degrees Celsius when actively browsing), although it takes a while for the gif to be downloaded. Ironically, my laptop cannot handle prime95 small FFTs torture test due to weaker thermal and physical design.

CPU: Ryzen 9 5950x; Undervolted using Core Optimizer setting in BIOS for even lower temperatures Motherboard; Gigabyte x570 Aorus Master rev 1.2 BIOS version F33 CPU cooler: EK AIO 360 D-RGB CPU cooler runs at max speed when CPU temperature is 75 degrees Celsius

[leoding86]

Simple explanation is that generating GIF is a high CPU-sensitive job in the extension, so it's normal.

Complex explanation is that because the browser extension is created using javascript, so the generation feature is implemented by javascript too. For doing this kind of job, javascript isn't a good performance program language. And, for the speed of generating a GIF, there are 4 workers are working in background while generating a GIF, this will make the CPU usage higher.

[Pxartist]

Simple explanation is that generating GIF is a high CPU-sensitive job in the extension, so it's normal.

Did you fully read my report? Your gif generator puts even more strain than the some of the most popular and highly regarded CPU stress test benchmarks. If anything, my report should be indicative that your implementation of this feature scales poorly on higher end systems. The temperature discrepancy is ridiculous.

If you do not wish to use a more efficient programming language, at the very least introduce a feature that allows the user to adjust the intensity of the GIF generation process so higher end systems won't attempt to kill themselves over such a task.

With that aside, the current state of your GIF generation feature has the potential to be the next greatest CPU stress test benchmark with a few changes. In fact, depending on the circumstances, it may serve to be a powerful weapon against the even higher end server CPUs that have even higher core counts than my PC.

[leoding86]

I kwon what's your meaning. The library of generating GIF I used is gif.js, you can go to the site https://jnordberg.github.io/gif.js/. There is a demo you can play. When you change the settings, the GIF will be re-generated, and you'll notice the CPU usage will become high. There are few frames used and use only 2 workers in background to do the job.

So, as I said, it's normal. The only thing that I can do is give a setting to allow the user to change the number of workers can be spawned. But it will affect the speed of generating GIF significantly.

[Pxartist]

The library of generating GIF I used is gif.js, you can go to the site https://jnordberg.github.io/gif.js/. There is a demo you can play. When you change the settings, the GIF will be re-generated, and you'll notice the CPU usage will become high. There are few frames used and use only 2 workers in background to do the job.

None of the demos make my CPU temperature go to 85+ degrees Celsius. The highest temperature I've seen with playing around with the demos is 55 - 65 degrees Celsius. I am not concerned with the CPU usage. I am concerned with how high my CPU temperature is when I attempt to generate a GIF, which should not be more intensive than a prime95 small FFT torture test.

您的程式的"Generate GIF"的功能給高階的電腦可以跟電腦病毒比. 產生GIF檔的壓力不應該多於 prime 95 small FFT torture test - CPU產生的熱度也不應該那麼大. 如果您不修改這個問題, 您會害死很多人的CPU. 您看起來不知道什麼是prime95 和 CinebenchR23 https://henryshu.pixnet.net/blog/post/47968836 https://www.pcdvd.com.tw/showthread.php?t=1179869

您的程式的"Generate GIF"的功能给高阶的电脑可以跟电脑病毒比. 产生GIF档的压力不应该多于prime 95 small FFT torture test - CPU产生的热度也不应该那么大. 如果您不修改这个问题, 您会害死很多人的CPU. 您看起来不知道什么是prime95 和 CinebenchR23 https://henryshu.pixnet.net/blog/post/47968836 https://www.pcdvd.com.tw/showthread.php?t=1179869

I will no longer be using the "generate GIF" feature. 我以後不會用 "generate GIF" 的功能了. 我以后不会用 "generate GIF" 的功能了.

[leoding86]

因为DEMO只是用了少数几张图片，所以以你的CPU只需要工作很短的时间就可以完成工作，所以反应不出来温度变高。以我的机器而言，使用的是非常老旧的CPU，i5-2540m，因为性能的问题，依旧需要工作一段时间完成生成工作，所以温度依旧会变得比较高，如果是在Pixiv上以现有的worker数量生成GIF，会导致CPU使用率长期维持在100%。

这个不是在我这个方面能解决的问题，使用javascript生产图片的操作会非常消耗系统算力，因为是软算没有任何硬件加速。

至于你说不在用了，那是你的选择。但是你可以试试生产apng的操作，因为只使用了一个worker，CPU的使用率应该会运行在一个相对较低的水平。这就是为什么我说我能做到的就是提供一个调整worker数量的设置。

补充一点：CPU在不考虑到变频的情况下，使用率越高通常温度是越高的。

[leoding86]

不过我找到一个有意思的库gpu.js，可以是的js使用硬件加速。不过这个有些超出我的理解能力，需要花一些时间来学习和做一些库的移植工作。

[Pxartist]

补充一点：CPU在不考虑到变频的情况下，使用率越高通常温度是越高的。

When I run prime95 Small FFT torture test, my CPU utilization is 100% and my temperatures, as I have said earlier, are 72 - 75 degrees Celsius. On the other hand, "Generate GIF" results in a utilization of 25% (4 workers; 1 worker per core) and results in instantaneous sky high temperatures of 85+ degrees Celsius. A higher CPU utilization does not mean higher temperatures - the type of work being performed plays a big role in energy consumption and heat output.

There is a reason why benchmarking softwares have varying component temperatures even when at 100% load - the type of workload is different. If load always correlated with temperature, there would not be so much benchmarking software to begin with. This is why prime95 is highly regarded in terms of testing system stability and temperature - it's workload is by far some of the most intense and outrageously unrealistic in the workplace scenario.

因为DEMO只是用了少数几张图片，所以以你的CPU只需要工作很短的时间就可以完成工作，所以反应不出来温度变高。

Unlike "Generate GIF", my temperatures do not sharply jump to 85+ degrees Celsius. Maybe it is because the number of frames in the demo are too low, but when I use "Generate GIF" on ugoira with a fairly low frame count, I still end up with 85+ degrees Celsius.

For this, I can offer a number of reasons: the JavaScript library is likely poorly optimized and does not scale well with stronger systems like mine. Furthermore, based on the demo shown, it appears that the execution is very poor and it should not be used on high image count animations, such as pixiv ugoira. Likewise, it could be a limitation of JavaScript or a programming error as well.

I suggest that you utilize a different library, if not a different language, such as C++, or use a third party open source tool, such as ffmpeg, to create the gif. https://medium.com/@Peter_UXer/small-sized-and-beautiful-gifs-with-ffmpeg-25c5082ed733 https://unix.stackexchange.com/questions/24014/creating-a-gif-animation-from-png-files https://medium.com/@Peter_UXer/small-sized-and-beautiful-gifs-with-ffmpeg-25c5082ed733 https://gist.github.com/protrolium/21ab48468470ea8e3a72567fd8938abe https://mattj.io/posts/2021-02-27-create-animated-gif-and-webp-from-videos-using-ffmpeg/

If you were to switch the codebase from JS to C++, I would greatly applaud your efforts.

I am not the most familiar with Chrome extensions and the API, but such implementation should be possible.

I took a closer look at how Pixiv implements ugoiras via Inspect Element and have found that main server does in fact, send a zip file and a large number of image blobs. As to how you were able to intercept and catch them to download, I do not know yet, as they cannot be accessed directly via url. I have yet to try accessing them using command line based tools, such as wget.

以我的机器而言，使用的是非常老旧的CPU，i5-2540m，因为性能的问题，依旧需要工作一段时间完成生成工作，所以温度依旧会变得比较高，如果是在Pixiv上以现有的worker数量生成GIF，会导致CPU使用率长期维持在100%。

I do not care about the CPU utilization percentage. If you have not realized yet, this whole thread was created in concern to CPU temperature, not utilization. The temperature issue seems to be the mostly based in CPUs with high core count, especially those with 16+ cores and 32+ threads. My laptop has 6 cores and 12 threads, which may be the reason why I do not have such high temperatures when using the "Generate GIF" function (although, I also have undervolted it and lowered the TDP from 45 W to 35 W).

https://ark.intel.com/content/www/us/en/ark/products/50072/intel-core-i5-2540m-processor-3m-cache-up-to-3-30-ghz.html

Your usage of a 2 core, 4 thread CPU with a TDP of 35 W may be the reason why you have never encountered such issues to begin with. While your financial circumstances may not be the best, please consider getting a newer laptop / computer to gain a better understanding on how your program runs on modern and more popular systems.

不过我找到一个有意思的库gpu.js，可以是的js使用硬件加速。不过这个有些超出我的理解能力，需要花一些时间来学习和做一些库的移植工作。

Don't bother using GPU acceleration on that junk library. Even when performing 3D rendering via Blender without hardware acceleration and x264 encoding for 4K video (both for benchmark purposes), my CPU temperature will never be as high as doing a "Generate GIF" benchmark.

Consider trashing gif.js. I think that it is garbage and scales poorly with more complex (higher image count) workloads and scales even more poorly on higher core count systems. Look at the commit activity https://github.com/jnordberg/gif.js/graphs/contributors?from=2013-05-19&to=2021-07-23&type=c

, the last single (and very lonely and probably irrelevant) commit was on October 2019. The bulk of the activity on the repository took place from 2013 to 2015, which makes it very likely that the contributors never took into account of higher core count systems, especially the Ryzen 3000 Zen 2 architecture breakthrough (July 7, 2019) in much higher core counts and processing power. Hence why it has proven to be very damaging to my system.

TLDR; gif.js repository is very outdated and is not designed for modern, higher core count processors. It may even be poorly designed for AMD based systems. Please consider using a different library, especially one that is not JS. I do not like JS.

Chinese is my second language.

[leoding86]

Browser extension can't use any program language except javascript or use any external tool.

I noticed that you have 4 cores which is at high temperature which are matched the number of workers, and it seems right, each worker is spawned by browser will run at one core. I'm not very aware hardware, so there isn't much I can do to reduce the core temperature. Maybe slow down the generate process is a quick solution (add a sleep time between each loop).

Replace the library with a better one seems like a better solution, but it requires some time to find the one, especially I need taking care some personal matters, but I'll gave it a try.

I'm studying english and not very good at it.

[leoding86]

I think I found a solution for resolve the issue. It's technical based on webassembly. I need to investigate more details and give it a try. We can wait and see.

[leoding86]

I tried to use ffmpeg to convert ugoira to gif, and I found that the results are different between different browsers. In Edge, ffmpeg needs more CPU to complete the job and is slower than gifjs. But in Chrome, ffmpeg has a similar speed compared to gifjs, but uses less CPU.

I think there is no certain way to solve this issue. I'll leave this open, anyone have any thought could leave a comment here.

[Pxartist]

In Edge, ffmpeg needs more CPU to complete the job and is slower than gifjs. But in Chrome, ffmpeg has a similar speed compared to gifjs, but uses less CPU.

Speed shouldn't be a primary concern unless the speed difference is very large. The main issue at hand is whether or not the said feature is going to be damaging the hardware/CPU. If the marginal speed requires the expense of significant component lifespan, it is simply not worth it (very much like excessive CPU overclocking; you can make a consumer grade CPU faster than a server CPU if you wish to make it run very hot and last for a small fraction of its lifespan).

Differences in CPU consumption and speed when using different browsers should also be expected, as each browser will likely function differently internally. There is a reason why "Browser benchmarks" exist.

Furthermore, I would like to remind you that you are using a 10 year old CPU and likely do not have a full picture of the differences.

If you push the code to a test branch and create a functional beta release of pixiv toolkit that uses ffmpeg, I would be willing to test it for you.

[leoding86]

I published a preview version 5.0.0. There are some new features in the version include the feature using ffmpeg to convert ugoira to gif. You can install it using developer mode. Here is the download page

[Pxartist]

I have tested version 5.0.0 using ffmpeg and it works very well. The gif generation time is much, much shorter and only has a total CPU % Utilization of 5 - 7%. The temperature has drastically dropped to the point that it is lower than that of generating apng and webm (60 - 65 degrees Celsius vs 65 - 75 degrees Celsius).

The first large peak was the generation of a webm and apng file. The second and third pointy peaks was the process of generating a gif file. The width of the peaks indicates how long each process took.

I also did a quality comparison using the old JS library vs the ffmpeg library:

gif generated with default JS library is on the left and gif generated with ffmpeg is on the right.

100% Zoom (default size)

300% Zoom

At first glance, there does not appear to be any differences, but upon closer inspection, the ffmpeg file appears to have more noise.

This can easily be fixed by adjusting the ffmpeg parameters. If possible, could you add in an advanced tab for the ffmpeg option to allow the user to manually configure the parameters?

Anyhow, with such results, you may want to consider utilizing ffmpeg further and may achieve even better results by using it for apng and webm generation.

[leoding86]

The webm and apng are also generated by ffmpeg when you select ffmpeg as current converter. I'm not very similar with ffmpeg, but I found a solution for convert better quality animation with this comment in my script:

this.runFFmpeg('-r', this.frameRate, '-i', '%06d.jpg', '-vf', `fps=${this.frameRate},scale=${this.width}:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse`, '-plays', 0, 'out.gif');

But the part flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse is acting like a black magic to me. What are [s0], [s1], [p] for? I don't have any clue 😕. And the documentation of ffmpeg is too long.

[Pxartist]

The documentation is not very long if you use Ctrl+F (Find...) and look for the desired terms at https://ffmpeg.org/ffmpeg-all.html using terms such as split, palettegen, or paletteuse. flags= is effectively providing additional parameters/flags for ffmpeg to use.

lanczos is an algorithm and appears to be usable under section 39.264.1 (39.264 zscale) and 29 Scaler Options.

split documentation can be found under section 44.29 split, asplit. If a number is not specified, 2 separate but identical video stream outputs are generated from a single input. Thus, [s0] == [s1], effectively speaking [s0] and [s1] are effectively variables storing duplicate video streams.

palettegen documentation can be found under 39.165 palettegen. palettegen generates a palette from an input video stream, which then can be used for the paletteuse command. The ';' indicates a separate command line, which is why [s0] is needed in front of palettegen. Similar to split, a variable, [p], is used to store the palette png file.

paletteuse documentation is just under palettegen and is under section 39.166. paletteuse takes in an input video stream and a 256 pixel png image (ideally generated from palettegen) to output a downscaled video stream. The ';' indicates a separate command line, which is why [s0] and [p] is needed in front of paletteuse.

Although I have yet to test it, it may be possible that paletteuse and palettegen cannot both be on the same line, which is why the ';' are needed to split up the procedure. Theoretically speaking, using [s0][p]paletteuse instead of [s1][p] should yield the same result. https://superuser.com/questions/1275517/how-do-i-use-palettegen-and-paletteuse-on-the-same-command-with-ffmpeg https://www.reddit.com/r/ffmpeg/comments/ajs0pt/palettegenpaletteuse_in_the_same_string_with/

[Pxartist]

Hey could you add ffmpeg to the Firefox version as well?