search

dimroc / etl-language-comparison

Count the number of times certain words were said in a particular neighborhood. Performed as a basic MapReduce job against 25M tweets. Implemented with different programming languages as a educational exercise.
http://blog.dimroc.com/2015/11/14/etl-language-showdown-pt3/
187 stars 33 forks source link

Include memory consumption in the benchmarks #23

Open fervic opened 8 years ago

fervic commented 8 years ago

Hi @dimroc, nice repo and blog posts!

It would be nice to also see a memory consumption comparison. For example the Parallel Ruby implementation got close in speed to the Go sub-string implementation, but Go's goroutines are supposed to have a much lower memory consumption than spawning multiple copies of the Ruby program, so it would be good to see this shown here.

dimroc commented 8 years ago

That is a great point. Go is exceptional at memory management. I'll be sure to address that in my next blog post. If you know of any good CLI tools to measure the memory of a process, and it's subprocesses, let me know. I scoured here (http://unix.stackexchange.com/questions/18841/measuring-ram-usage-of-a-program).