search

Kyligence / kylin-tpch

Run TPCH Benchmark on Apache Kylin
22 stars 22 forks source link

[Report] SSB Benchmark report of Kylin 4.0.0 #7

Open hit-lacus opened 3 years ago

hit-lacus commented 3 years ago
hit-lacus commented 3 years ago

1

hit-lacus commented 3 years ago

2

hit-lacus commented 3 years ago

3

hit-lacus commented 3 years ago

4

hit-lacus commented 3 years ago

Kylin Configuration

kylin.metadata.url=ssb_kylin400@jdbc,url=jdbc:mysql://ip-XXX.cn-northwest-1.compute.internal:3306/hive,username=hive,password=XXX,maxActive=10,maxIdle=10,driverClassName=org.mariadb.jdbc.Driver
kylin.env.zookeeper-connect-string=ip-XXX:2181

## Disable retry
kylin.engine.max-retry-time=1

## Build Engine Resource
kylin.engine.spark-conf.spark.executor.cores=2
kylin.engine.spark-conf.spark.executor.instances=25
kylin.engine.spark-conf.spark.executor.memory=7GB
kylin.engine.spark-conf.spark.executor.memoryOverhead=1GB

## Query Engine Resource
kylin.query.spark-conf.spark.master=yarn
kylin.query.spark-conf.spark.driver.cores=1
kylin.query.spark-conf.spark.driver.memory=8GB
kylin.query.spark-conf.spark.driver.memoryOverhead=1G
kylin.query.spark-conf.spark.executor.cores=2
kylin.query.spark-conf.spark.executor.instances=30
kylin.query.spark-conf.spark.executor.memory=8G
kylin.query.spark-conf.spark.executor.memoryOverhead=2G
kylin.query.spark-conf.spark.sql.parquet.filterPushdown=false
kylin.query.spark-conf.spark.sql.files.maxPartitionBytes=64MB

## Disable canary
kylin.canary.sparder-context-canary-enabled=false

## Shard setting
kylin.storage.columnar.shard-size-mb=50
kylin.storage.columnar.shard-rowcount=600000
kylin.storage.columnar.shard-countdistinct-rowcount=300000
kylin.storage.columnar.repartition-threshold-size-mb=50
kylin.engine.spark-conf.spark.hadoop.parquet.block.size=268435456