fraganator / archive-cache-manager

A LaunchBox plugin which extracts and caches large ROM archives, letting you play games faster.
GNU Lesser General Public License v2.1
11 stars 5 forks source link

Improve file priority performance for many priorities #24

Closed fraganator closed 1 year ago

fraganator commented 2 years ago

When an archive isn't cached, each file priority is checked by calling 7z.exe with a wildcard match for that priority. If there are many file priorities to be checked, 7z.exe is called multiple times, and the overhead of repeatedly running the process becomes noticeable.

The plugin previously listed all the files with a 7z l call, then operated on that list in memory. This was modified when addressing #10, where reading tens of thousands of files over stdout had a large performance impact. This was partly resolved by moving the file priority matching to a 7z command line argument, so the result would be a small subset of files.

Look at moving wildcard matching back to plugin, at least in case where there are many priorities to check. Two wildcard matching options: