[Report] TPC-H Benchmark report of Kylin 3.1.2

hit-lacus commented 3 years ago

Table of Content

hit-lacus commented 3 years ago

Summary

The background information are record at Benchmark step of Kylin 3.1.2

Summary

Metrics Name	Metrics Value
Scale Factor	10
Data Load Duration	287.75 minutes
Storage Size	960.033 GB
Query RT(Total)
QPS	0.08

Basic Cluster Info

Key	Value
Node Memory	64GB
Node Core	16
Node (Instance Type)	m5.4xlarge
Node Disk	1500GB + 100GB(SSD)
Node Num	4 Worker + 1 Master
EMR Version	emr-5.31.0
Kylin Version	3.1.2
Yarn Memory	204.80 GB
Yarn Core	52
HBase Memory	112 GB (28*4)
HBase Core	?
HDFS Capacity	5.83 TB

hit-lacus commented 3 years ago

Data Load Duration

Cube	Value
lineitem	248.63 min
partsupp	18.85 min
customer_vorder	16.05 min
customer	4.22 min

hit-lacus commented 3 years ago

Storage Size

Cube	Value
lineitem	949.046 GB
partsupp	9.28809 GB
customer_vorder	1.65234 GB
customer	0.047 GB

hit-lacus commented 3 years ago

Response Time

hit-lacus commented 3 years ago

Kylin Configuration

## Use JDBC Metadata instead of HBase please ...
kylin.metadata.url=benchmark_kylin312@jdbc,url=jdbc:mysql://ip-XXX.cn-XXX-1.compute.internal:3306/hive,username=hive,password=rAgNAstbqF33pqFe,maxActive=10,maxIdle=10,driverClassName=org.mariadb.jdbc.Driver

## Query Cache 
kylin.query.cache-enabled=false

## MR related
#kylin.engine.mr.config-override.mapreduce.map.java.opts=-Xmx6g
#kylin.engine.mr.config-override.mapreduce.map.memory.mb=7000
#kylin.engine.mr.config-override.mapreduce.reduce.java.opts=-Xmx10g
#kylin.engine.mr.config-override.mapreduce.reduce.memory.mb=11000

## Hive CLI related
kylin.source.hive.config-override.mapreduce.map.java.opts=-Xmx7g
kylin.source.hive.config-override.mapreduce.map.memory.mb=8192
kylin.source.hive.config-override.mapreduce.reduce.java.opts=-Xmx12g
kylin.source.hive.config-override.mapreduce.reduce.memory.mb=13000
kylin.source.hive.config-override.tez.task.resource.memory.mb=13000
kylin.source.hive.config-override.hive.tez.container.size=13000

## dfs.replication
kylin.engine.cuboid.dfs.replication=1

## HBase related
kylin.storage.hbase.region-cut-gb=3
kylin.storage.hbase.hfile-size-gb=1.5

hit-lacus commented 3 years ago