Right now we generate all files in the same directory. This is great if you have a few dozen to a few hundred images.
Depending on your server's filesystem, you might hit a performance roadblock after putting a few thousand (typically around 3,000) files in a single directory. Even if the server's filesystem is tuned to use b-tree hashes for the directory listing, PHP uses readdir() which only processes 32K of directory index data at a time, making access to densely populate folders extremely slow.
The best approach is to use 256 subdirectories in the output directory where the directory name is the last two hex digits of the MD5 hash of its filename. This greatly reduces the number of files and works very well up to the point you hit a few million images.
Right now we generate all files in the same directory. This is great if you have a few dozen to a few hundred images.
Depending on your server's filesystem, you might hit a performance roadblock after putting a few thousand (typically around 3,000) files in a single directory. Even if the server's filesystem is tuned to use b-tree hashes for the directory listing, PHP uses
readdir()which only processes 32K of directory index data at a time, making access to densely populate folders extremely slow.The best approach is to use 256 subdirectories in the output directory where the directory name is the last two hex digits of the MD5 hash of its filename. This greatly reduces the number of files and works very well up to the point you hit a few million images.