Closed coyotemarin closed 4 years ago
Probably should add the following options:
docker_image
: name of the docker imagedocker_registry
: hostname of the docker registry (defaults to centos
, which is Docker Hub)docker_client_config
: the config file on HDFS, needed to log into ECRdocker_mounts
: a list of volumes to mount inside docker. Defaults to ['/etc/passwd:/etc/passwd:ro']
Magic mrjob can provide:
emr_config
cmdenv
We'll need to bootstrap mrjob in setup if we want to be able to access it inside Docker.
EMR now has explicit support for Docker on 6.x AMIs. mrjob should integrate with this