Improve Projections

[yingsu00]

Projections was one of the top contributor to memory allocations that causes full GCs and OOMs. It also has space to improve in terms of CPU usage. We propose the following changes to optimize projections:

Pre-allocate enough memory

• Part one was implemented in https://github.com/prestodb/presto/pull/15206 which brings 33% improvement in projection's throughput (higher is better) and 4x reduction in gc.alloc.rate.norm (lower is better) in GC benchmarks.

• Part two will be additionally avoiding unnecessary growCapacity for nested types.

Optimize BlockBuilder element insertions

• Currently each writeXXX go through several checks for each value. Some checks can be omitted.
• getBlock() takes high CPU usage because it's called on each value. This is totally unnecessary.
• isNull() takes high CPU usage. For blocks that can't have nulls this is totally unnecessary.

Create DictionaryBlock for identity projections

• Currently blocks are copied. Building DictionaryBlock would be much faster
• Then we probably need to disable MergingPageOutput before repartitioning because repartitioning would flatten the DictionaryBlock.

Optimize aggregation functions https://github.com/prestodb/presto/issues/15361

[yingsu00]

cc @bhhari @mbasmanova @sujay-jain @biswapesh

[yingsu00]

Add identity projection with partial filter failure rate to BenchmarkPageProcessor https://github.com/prestodb/presto/pull/15266

[yingsu00]

https://github.com/prestodb/presto/pull/15272 is out: Identity projection improvement when the selectedPositions is a list BenchmarkPageProcessor shows 21x improvment in throughput and 10x reduction in memory allocation and GC

[yingsu00]

pnb1_batch1_worker_cpu_20201011_012349_60s.svg.zip Profile shows some functions e.g. mapConact is the major CPU consumer for projections. These functions use TypedSet and were very inefficient in using PageBUilder and BlockBuilders. So it seems optimizing PageBuilder and BlockBuilders has a very wide benefit.

[yingsu00]

https://github.com/prestodb/presto/pull/15301 is out: Disable MergingPageOutput for ScanFilterAndProjectOperator before repartitioning. Simple query on TPCH 100GB shows 22% CPU reduction: