search

andyburgin / hadoopi

This project contains the configuration files and chef code to configure a cluster of five Raspberry Pi 3s as a working Hadoop running Hue.
http://data.andyburgin.co.uk
39 stars 17 forks source link

Question about Caveats - small RAM #9

Closed balazsdukai closed 5 years ago

balazsdukai commented 5 years ago

I read that the 1GB RAM is a real bottleneck in the setup. If your were to build the cluster again, would you choose a Raspberry Pi 4 with more RAM? I know that more is better ;-), but do you think that the increased RAM would justify the higher cost of the cluster?

Speed is not much of an issue I guess, it's more like if more RAM per Pi would allow to run things that are currently not possible? If so, then the 2GB or 4GB model? :-)

andyburgin commented 5 years ago

Please first see my reply to https://github.com/andyburgin/hadoopi/issues/8

The Pi4 with extra ram would certainly help but you need more powerful power supplies than the ones I used, more importantly the Pi 4 needs to use the latest version of raspbian - you are going to hit a whole bunch of dependency issues compiling hadoop and the other components

balazsdukai commented 5 years ago

Again, thank you for your reply! As you mentioned, I would like to learn how to run things on hadoop, as opposed to learning how to maintain hadoop itself, and resolving the dependency issues certainly no something that I enjoy doing.