Use synchronous globbing of npm 2 packages

[lime]

As mentioned in #32, it looks like glob.sync uses significantly less memory than the async glob, and it should be faster as well.

The changes in 28a1bdbf2b803e77027f5aa5840fadeff516345f did a lot to improve lookup performance, but the swap from sync to async should be safe to revert.

I'll see if I can whip up some benchmarks to show the difference.

[JamieMason]

great, thanks a lot @lime I'll get this merged in tomorrow. Thanks for giving up some time to help.

[lime]

I made a gist where I tracked resource usage of a run using Syrupy. As you can see, memory usage gets pretty insane before getting cut off by (I assume) the default memory limit in node. :) This is on node 4.4.0 and npm 2.14.20, in a repo with ~600 tarballs.

[JamieMason]

I've formatted them side-by-side here. If I understand this correctly: it took 22 seconds to complete vs 1:31? and used around half as much memory? that's a huge win, nice work!

ELAPSED    CPU    MEM | ELAPSED    CPU    MEM
  00:00    0.0    0.0 |   00:00    0.0    0.0
  00:01    2.7    0.3 |   00:01    3.1    0.3
  00:02    0.0    0.3 |   00:02    0.0    0.3
  00:03    0.0    0.3 |   00:03    0.0    0.3
  00:04    0.0    0.3 |   00:04    0.0    0.3
  00:05   92.4    0.6 |   00:05   93.8    1.1
  00:06   96.1    0.7 |   00:06  108.0    2.0
  00:07   95.5    0.7 |   00:07  102.9    2.5
  00:08   95.2    0.8 |   00:08   96.8    2.8
  00:09   79.9    1.0 |   00:09  106.8    3.9
  00:10   97.8    1.2 |   00:10  102.8    4.8
  00:11   99.6    1.3 |   00:11  104.5    4.9
  00:12   96.0    1.4 |   00:12  102.9    4.9
  00:13   14.2    1.4 |   00:14  103.8    5.4
  00:14    4.7    1.4 |   00:15  100.4    6.4
  00:15    3.2    1.4 |   00:16  101.8    7.4
  00:16    3.3    1.4 |   00:17  108.1    7.9
  00:17    0.3    1.2 |   00:18  102.1    8.9
  00:19    0.9    1.1 |   00:19  100.6    9.5
  00:20    5.5    1.1 |   00:20  100.4   10.2
  00:21    4.7    1.2 |   00:21  104.4    9.6
  00:22    0.0    0.0 |   00:22  101.2    9.7
  00:23  101.2    9.7 |
  00:24  100.0    9.8 |
  00:25  101.1    9.9 |
  00:26  102.1   10.5 |
  00:27  102.4   11.2 |
  00:28  101.9   12.7 |
  00:29  113.0   12.9 |
  00:30  105.4   13.0 |
  00:31  105.5   13.1 |
  00:32  104.3   13.1 |
  00:33  102.4   13.9 |
  00:34  114.0   13.4 |
  00:35  103.2   13.4 |
  00:36  106.2   13.4 |
  00:37  104.0   13.4 |
  00:38  104.6   13.5 |
  00:39  106.7   13.5 |
  00:40  100.6   13.5 |
  00:41  119.7   13.2 |
  00:42  102.2   14.3 |
  00:43  118.4   15.0 |
  00:44  109.6   15.2 |
  00:45  109.2   15.3 |
  00:46  102.6   15.3 |
  00:47  101.5   15.3 |
  00:49  103.1   15.3 |
  00:50  101.4   15.3 |
  00:51  103.2   15.3 |
  00:52  100.8   15.1 |
  00:53   98.6   14.1 |
  00:54  104.5   13.5 |
  00:55  104.1   13.8 |
  00:56  104.1   14.3 |
  00:57  101.5   14.9 |
  00:58  101.3   15.2 |
  00:59  108.8   15.5 |
  01:00  101.9   16.5 |
  01:01  100.0   16.8 |
  01:02   99.8   17.0 |
  01:03  114.4   17.2 |
  01:04  111.4   17.2 |
  01:05  101.6   17.1 |
  01:06  105.4   16.9 |
  01:07  103.9   16.9 |
  01:08  100.7   16.9 |
  01:09  101.3   16.9 |
  01:10  101.5   16.9 |
  01:11  102.7   16.9 |
  01:12  100.7   16.9 |
  01:13  105.1   15.9 |
  01:14  102.4   15.9 |
  01:15  102.8   16.0 |
  01:16  100.5   16.0 |
  01:17  111.7   15.7 |
  01:18  101.5   16.7 |
  01:19  100.0   17.1 |
  01:20  100.4   17.3 |
  01:21  100.3   17.9 |
  01:23  100.0   17.9 |
  01:24  106.9   17.7 |
  01:25  105.3   17.7 |
  01:26  118.4   17.6 |
  01:27  106.3   17.6 |
  01:28  104.4   17.6 |
  01:29  103.0   17.6 |
  01:30  101.3   17.5 |
  01:31    0.0    0.0 |

[lime]

Yyep. 💃 Can't say for sure whether this would happen on other machines as well, but in my case it was a pretty massive difference.

I also tried plotting two earlier test runs with gnuplot 📈 :

[JamieMason]

Released in 0.12.2, thanks @lime.