Hadop MapReduce 架构和流程

[liusheng]

经典的MapReduce过程 Input: 一系列key/value对 用户提供两个函数实现: map(k,v) ->list(k1,v1)【(k1,v1) 是中间key/value结果对】 reduce(k1, list(v1)) ->v2 Output：一系列(k2,v2)对

内部裸机

外部物理架构

参考

https://blog.csdn.net/dingguanyi/article/details/78254660

[liusheng]

https://www.netjstech.com/2018/04/data-compression-in-hadoop.html https://www.netjstech.com/2018/02/how-mapreduce-works-in-hadoop.html https://www.netjstech.com/2018/04/how-to-compress-intermediate-map-output-hadoop.html https://www.netjstech.com/2018/05/input-splits-in-hadoop.html

介绍MapReduce流程好的文章： https://blog.csdn.net/dingguanyi/article/details/78254660 https://juejin.im/post/6844903783793704967