Question about White Elephant and M/R

LinkedInAttic / white-elephant

Hadoop log aggregator and dashboard

Other

191 stars 63 forks source link

Question about White Elephant and M/R #7

Closed pbj0001 closed 11 years ago

pbj0001 commented 11 years ago

Is it correct that the White Elephant log uploader only works with M/R 1 and not Yarn, since statsupload.pl appears to look only in the local file system? I believe Yarn stores the history files in HDFS already.

Can White Elephant as it is currently implemented handle Yarn logs already in HDFS?

matthayes commented 11 years ago

I haven't tested White Elephant against Yarn log files yet. So not sure if it can handle the log files stored directly in HDFS.