search

d2iq-archive / dcos-flink-service

11 stars 17 forks source link

hdfs integration #22

Open skonto opened 7 years ago

skonto commented 7 years ago

Shouldnt core-site, hdfs-site files be downloaded to the taksmanagers as well provided a docker image is available for them (do we have one?), for example to use then the file in hdfs client code? If the files are updated should we restart the flink job? Same concept applies to Spark: https://docs.mesosphere.com/service-docs/spark/hdfs/

EronWright commented 7 years ago

I believe that the TM does download the Hadoop configuration files from the JM, via the JM's artifact server. The JM itself is provided the configuration files in the normal way, via the Mesos fetcher. @skonto are you seeing a problem related to this?