Open efeller opened 10 years ago
hi @efeller, I'm not sure you're aware but there is currently development going on Mesos to integrate Docker directly, so you won't need to launch Docker with External Containerizer + Deimos, but just Mesos. I'm also going to be putting a patch for Spark to make Spark support docker images natively as well, so you don't need to upload a spark tar but just specify a docker image. So with the new changes and the new API being proposed (MESOS-1593) you can specifiy docker options to enable things like that. Let me know if you have more comments, definitely like to learn more about your setup and what you need!
Thanks Timothy. Sounds great. I will have a look at the Mesos + Docker integration progress. In the meantime I have modified the deimos/mesos.py file (line 94) to make it work.
I was recently evaluating Spark + Mesos + Deimos and run into the following issue. By default the Mesos scheduler backend in Spark sets the command to be executed as "cd %s*; ./sbin/spark-executor". This command is passed to Docker run arguments as "sh -c cd ..". However in my setup to enable networking inside the containers, the Docker run arguments need to be prefixed with "dhclient eth0". It would be great if Deimos could provide an option (perhaps in the deimos.cfg) to add prefixes to the Docker run arguments (resulting in sh -c dhclient eth0; ...; cd ..).