Multithreading not working

[beakwood]

I don't know why. Multithreading is not working for me. Yes, I see the CPU only go up by like 8%, but it's nothing like I've been seeing previous users CPU usage % in the 95%+

I have tried B1-B5, and Multithreading is just not working for me. Windows 11 24H2 build, could be the problem. But, it's not working when selecting default, or even random thread count from 10 to 256.

I've tried CLI command and exe with Run As Admin ticked as well. Also, when I use command like WindowsThumbsPreloader.exe -m 10(or any n) -r it only opens the program and not preload. But if I remove the n from -m it works.

[Mfarooq360]

Hello, on Beta 5, do you have logging enabled? I'd like to check a log file to see if multithreading is actually being enabled properly or some other issue is occurring such as a specific file format causing a problem.

Yes, I see the CPU only go up by like 8%, but it's nothing like I've been seeing previous users CPU usage % in the 95%+

I also wonder what the disk usage is while running the program as multithreaded, since hard drives may max out their read speeds a lot faster than the cpu being fully utilized.

Also, this version of the program is a lot more optimized in multithreaded mode for CPU usage compared to previous forks which means the CPU usage will be a lot less while preloading much faster, which may cause drives to max out more easily or may otherwise cause a bottleneck elsewhere such as with Windows.

Also, when I use command like WindowsThumbsPreloader.exe -m 10(or any n) -r it only opens the program and not preload. But if I remove the n from -m it works.

As for the command line arguments, the number should be connected to the -m flag such as -m10 or -m:10 to set the number of threads.

[beakwood]

Thanks, I will give the program another try and also use correct -m command.

Here's a log along with a screenshot of task manager. Looks like one CPU is being used while others are slightly used.

WinThumbsPreloaderLog_20240509.txt

[VL4DST3R]

It does indeed work on B5 for me

but as Mutahar said, check to see if you're not capped by something else. I also don't specify the number of threads at all when using -m.

[beakwood]

@VL4DST3R I'm leaning more into the fact that it could be very well be my Windows 11 / 24h2 26100.1 build. As I'm also noticing, I'm having a memory leak issue with Icaros, consistently rises before it crashes.

[beakwood]

@Mfarooq360 Just installed Windows 11 LTSC and one CPU is being maxed out while disk is also doing 100% but my thread count and cpu usage is nowhere near as @VL4DST3R

[VL4DST3R]

Have you tried to generate it on another divice? From your task manager it shows you're doing this on an usb external device, maybe it is just slow? Try on a regular sata/m.2 ssd and see if there's any difference in speed.

[VL4DST3R]

This is bordering o grasping at straws territory but all I can think of is:

• Either you have one of the new E and P core config cpu - maybe it doesn't play nice with multithreading in that case?

• Or secondly, try spinning up a windows 10 VM and see if it behaves differently there. I can help test the second one in the opposite config (w10 host with w11 vm) and see if it is indeed related to windows 11, although AFAIK Mutahar is also using w11 so it would make sense he tested on that first.

[Mfarooq360]

I am also wondering how this is happening as the logs state that multithreading is enabled with 255 threads, I initially thought that perhaps the program wasn't actually enabling multithreaded mode, but it seems to work for @VL4DST3R, so it may be Windows or perhaps Icaros is causing problems.

Eyther you have one of the new E and P core config cpu - maybe it doesn't play nice with multithreading in that case?

Looks like the CPU being used is an i9-10850K which uses regular cores, though this is also grasping at straws, but perhaps the core affinity was manually set, though I doubt it.

[VL4DST3R]

he core affinity was manually set, though I doubt it.

Yeah, something like that may be at fault. Any funky settings within bios @beakwood?

[VL4DST3R]

Also another thing that came up while discussing this issue with Mutahar - what kind of files are you generating thumbs for @beakwood?

[beakwood]

Some good news. I did try another OS build. This time around, I tried a custom OS, Ghost Spectre. I have seen the CPU working a bit more and files were being processed at a good 30-50 files. Utilization jumped up to about 46% and some CPU's we're slightly below half usage per this single directory.

I did come across the issue thread regarding @Mfarooq360 perhaps providing a BULK option. As this was what I was doing a majority of the time as well. For example, I have a folder with many subdirectories and including all the files is 354997 in total. When using winthumbpreloader on parent directory with -r, it only will process the files 3 at a time. But, as I suspect, it's limitation in the way winthumbs processes such large sum files and directories.

I also did a few other tweaks that may or may not have helped. For example, I disabled Storage Sense and unchecked delete temporary thumbnail cache. I also used winaero tweaker, and increased the thumbnail cache to 65535 KB/s. I also updated my bios. But, I did all this along with updating my bios on LTSC as well before using this custom build, and it didn't make a difference. But, it does seem to work.

Also, my extensions are as follows avif,bmp,gif,heic,heif,jpg,jpeg,mkv,mov,mp4,png,svg,ts,tif,tiff,webp

I am having a problem with Backup. I had to manually create the Explorer Backup directory manually. Since I was getting an error of it not finding it. But, it's not backing up anything in the backup when I click backup.

Edit:

RunAsAdmin - helped fixed the backup not backing up.

[Mfarooq360]

@beakwood Can you send a log from the program on the version of Windows where the multithreading works? I'd like to see if there's any difference made, although it could also be an internal difference or problem within either Windows or the program.

I'd also like to know what version of Windows that the custom install is using (like 23H2) in case the problem is due to a more recent version of Windows.

[beakwood]

@Mfarooq360 Windows 11 Pro 23H2 - 22631.3447 I currently have it preloading the large directory I spoke of regarding the 300K files, as it's at 70% now with 104450 files remaining. I'm not sure if I try preloading another directory will it slow it down the current preloading. But, I will most certainly grab a log for you.

[beakwood]

@Mfarooq360 I tested another directory with 20+K files using debug log. Then, when it was done, I preloaded using 1100 files using the All Logging, and it showed at least 20 thread was being used in the log. WinThumbsPreloaderLog_20240511.txt

[beakwood]

@Mfarooq360 Yes, it seems as though it's not able to preload recursively when it's more than 100 or so subdirectories. Yesterday and Today, 300K files between 1500 subdirectories, that are not being backup & preloaded. It takes 6 hours to complete and at the end. Winthumbspreloader does not save a backup. I've been testing highlighting 15 directories at a time and right-clicking and winthumbs recursively. But, even just 15 highlighted doesn't seem to back up correctly either. Which then raises the question, is there a way for winthumbspreloader, to start from top of the directory and work it's way down, one at a time?

[Mfarooq360]

@beakwood Sounds like the Windows cache reset issue which automatically resets the cache when it gets too big. I did notice that in the previous log you sent me, the backup appears to be 4.5GB in size so the cache is probably not doing too well. We're still looking for a way to prevent Windows from resetting the thumbnail cache, though so far we haven't found a concrete fix yet.

Which then raises the question, is there a way for winthumbspreloader, to start from top of the directory and work it's way down, one at a time?

I believe it does do that as it scans the subdirectories in a folder before moving onto the subdirectories of the subdirectories, which it then preloads in alphabetical order.

[beakwood]

@Mfarooq360 Thank you, appreciate everything!

[VL4DST3R]

@Mfarooq360 Yes, it seems as though it's not able to preload recursively when it's more than 100 or so subdirectories. Yesterday and Today, 300K files between 1500 subdirectories, that are not being backup & preloaded. It takes 6 hours to complete and at the end. Winthumbspreloader does not save a backup. I've been testing highlighting 15 directories at a time and right-clicking and winthumbs recursively. But, even just 15 highlighted doesn't seem to back up correctly either. Which then raises the question, is there a way for winthumbspreloader, to start from top of the directory and work it's way down, one at a time?

Have you considered using a bat file to recursively go through the folders and call winthumbs on each without the -r flag? Basically handle the actual folder selection outside the program itself and see if it makes a difference. Also don't be tempted (like I was early on when I found this tool) to launch all the folder gens in parallel/at once, you can very quickly peg your whole machine and render it unresponsive.

[beakwood]

@VL4DST3R I never thought about doing it that way. Let alone, I don't even know what script commands I would need to put in the batch file to get it to run as described. Now, I have been using CLI and added folder paths limiting it to about 50 folders in the CLI line following calling winthumbs. As it seems limiting to about 50 directories seem to back up successfully. Although, I do re-run twice to ensure it get leftovers it may have missed. Now, I've also been testing "deny" special permissions' explorer cache folder. So windows doesn't delete automatically. This "seems" to be helping. But, I haven't gone BIG on cache size like @Mfarooq360 stated, with the cache rising as high as 4.5GB. I could try one more time, let it preload for six hours and hopefully it backups.

[beakwood]

Yes, it still crashed at the end and didn't backup, unforutnately.

2024-05-13 06:59:21.342: Total items count: 348217 2024-05-13 06:59:21.343: Preloader state: Processing 2024-05-13 06:59:21.350: Process priority set to Below Normal 2024-05-13 06:59:21.350: Thread count: 256 2024-05-13 06:59:21.350: Preloading thumbnails Use Debug Logging for more detailed information (Slows thumbnail generation) 2024-05-13 11:40:41.086: Cache check timer stopped 2024-05-13 11:40:41.086: Preloader state: Done 2024-05-13 11:40:41.086: Preloader has finished 2024-05-13 11:40:41.086: Instance completed, ending instance.

[Mfarooq360]

Was auto backup turned on in the cache form of the GUI or did it still just not backup at the end?

2024-05-13 11:40:41.086: Instance completed, ending instance.

I also wonder how it crashed since it does look like it states that the instance completed successfully.

[beakwood]

@Mfarooq360 It finished, yes. But, it didn't back up in explorer cache or winthumbspreloader cache. This is why I've been doing 50 directories at a time. Because, once it completes. It even shows in the dialog box, backing up.... then it closes. It cache TO BOTH explorer cache and winthumbspreloader cache. But, when I do straight starting from parent directory and recursive that way, it doesn't say it at the end. It just finishes and closes. And does not cache to winthumbs or explorer cache. That's exactly what it shows in the log above. It never says, back up complete.