infolab-csail / wikipedia-mirror

Makefiles that will download and setup a local wikipedia instance.
1 stars 2 forks source link

Dumping Stage #1

Closed fakedrake closed 10 years ago

fakedrake commented 10 years ago

On ashmore dumping is pretty fast (about 7000 pages/sec at batch 8). Unzipping is the bottleneck but it is quite fast too. I am doing

make sql-dump-parts

before doing

make sql-load-parts

But once that works seamlessly it should make no difference. I look upon my old makefile code with horror...