search

dimroc / etl-language-comparison

Count the number of times certain words were said in a particular neighborhood. Performed as a basic MapReduce job against 25M tweets. Implemented with different programming languages as a educational exercise.
http://blog.dimroc.com/2015/11/14/etl-language-showdown-pt3/
186 stars 33 forks source link

Rust implementation #14

Closed potatosalad closed 9 years ago

potatosalad commented 9 years ago

I put together a very simple Rust implementation based on Rust 1.0 which was announced yesterday.

Update 2015-05-20: Added the option to run the the Rust implementation using substring or regex mode.

I've never written anything in Rust before, so there are probably some nasty bad practices or inefficiencies present. Below are the average timings I'm seeing on my machine.

Hardware: MacBook Pro 2.6GHz i7 (quad core) with 16GB RAM and PCIe SSD (very similar to the hardware in the part 2 blog post)

Rust w/ Substring 2.4s
Rust w/ Regular Expression 19.1s
Erlang w/ binary pattern matching 2.5s
Ruby w/ Celluloid 31.6s
Ruby w/ grosser/parallel 8.6s
Elixir 13.3s
Golang w/ Substring 8.6s

I'd say getting results like 2.4s from a very, very novice Rust programmer (maybe 2 hours total without any prior Rust knowledge) is fairly impressive. It's definitely a language I will be using more in the near future.

dimroc commented 9 years ago

Thanks for all this @potatosalad. I'm trying to get it to run locally but unfortunately cargo is giving me a hard time:

1__bash

I've already tried updating the dependencies:

Cargo.toml

[package]
name = "mapreduce"
version = "0.0.1"

[dependencies]
getopts = "0.2.14"
regex = "0.1.41"

I noticed a thread about Rust 1.2 regex being broken. Is that related? https://news.ycombinator.com/item?id=10023413

potatosalad commented 9 years ago

@dimroc I just ran this again using rust 1.0.0 (the version for which I originally wrote this) and it ran perfectly, so I'd assume something broke between rust 1.0 and 1.2. I'll see if I can get it running again properly and let you know what I find.

dimroc commented 9 years ago

Thanks a lot.

On Wed, Sep 2, 2015 at 12:06 PM, Andrew Bennett notifications@github.com wrote:

@dimroc https://github.com/dimroc I just ran this again using rust 1.0.0 (the version for which I originally wrote this) and it ran perfectly, so I'd assume something broke between rust 1.0 and 1.2. I'll see if I can get it running again properly and let you know what I find.

— Reply to this email directly or view it on GitHub https://github.com/dimroc/etl-language-comparison/pull/14#issuecomment-137145819 .