I happened to have the mother of all parses sitting around -- 1.2B rows totaling around 430Gb -- so I thought I'd give vroom a whirl. The machine has 128GB RAM and 128 (64 real) cores. Typing x <- vroom('file') caused CPU usage hit 12800% so I know it was hitting all cores. It ran long enough without returning that I had to move on to other things. Just thought I'd pass it along in case anyone is interested. I could do more serious testing on request.
hadley
commented
1 year ago
I happened to have the mother of all parses sitting around -- 1.2B rows totaling around 430Gb -- so I thought I'd give vroom a whirl. The machine has 128GB RAM and 128 (64 real) cores. Typing
x <- vroom('file')caused CPU usage hit 12800% so I know it was hitting all cores. It ran long enough without returning that I had to move on to other things. Just thought I'd pass it along in case anyone is interested. I could do more serious testing on request.