Use LRU caching for avoiding disk I/O when doing directory traversal + file reading.

[papandreou]

May I humbly suggest accomplishing that using https://github.com/papandreou/node-memoizeasync wrapped around fs.readdir and fs.readFile?

If the LRU part is important, I can add that as a supported expiration scheme.

[msiebuhr]

Does sound good. A slim wrapper around stat() with IsFile() and IsDirectory() shortcuts would be nice as well.

[papandreou]

memoizeAsync@0.3.0 uses lru-cache to do the actual caching and exposes its options, so that should be possible using something like this:

var memoizeAsync = require('memoizeasync'),
    fs = require('fs'),
    memoizedFs = {
        readdir: memoizeAsync(fs.readdir, {
            max: 10000,
            length: function (err, results) {
                return results ? results.length : 5; // Arbitrarily count memoized errors as equivalent to 5 items
            }
        }),
        readFile: memoizeAsync(fs.readFile, {
            max: 40 * 1024 * 1024,
            length: function (err, body) {
                if (Buffer.isBuffer(body) || typeof body === 'string') {
                    // Should actually be Buffer.byteLength(body) for strings, but that would need to plow through the entire thing
                    return body.length;
                }
                return 512; // Arbitrarily count memoized errors as equivalent to half a kilobyte
            })
        }),
        stat: memoizeAsync(fs.stat, {max: 10000})
    };

Next up would be figuring out how to make glob use memoizedFs.

[papandreou]

memoizedFs.isFile = function (fileName, cb) {
    memoizedFs.stat(fileName, function (err, stats) {
        cb(null, !err && stats.isFile());
    });
});