search

Segence / docker-hadoop

A Docker container with a full Hadoop cluster setup with Spark and Zeppelin
63 stars 40 forks source link

Proposing some useful defaults #8

Open saif-ellafi opened 4 years ago

saif-ellafi commented 4 years ago

Hi,

Hope this is not very biased, but having two nodes by defaults makes much more sense for an initial state. Also, added three yarn settings that are useful to do any kind of experimentation by default.

I added a little bit "more" than yarn defaults as a base, but usually to get anything done with spark we still need to pump up those minimums (default is 1024M)

saif-ellafi commented 4 years ago

@robvadai do you know why when I call docker-compose it does not put my yarn-site changes on the image when loading it?

robvadai commented 4 years ago

hi @saif-ellafi , the image needs to be rebuilt and docker compose just uses the latest image tag. To test you need to build the image yourself locally. You can use the build-docker-image.sh script, just run it.