Improve filtering performance by using SIMD predication

manticoresoftware / columnar

Manticore Columnar Library

Apache License 2.0

82 stars 15 forks source link

Improve filtering performance by using SIMD predication #26

Open githubmanticore opened 1 year ago

githubmanticore commented 1 year ago

When columnar storage processes a filter, there are two heavy stages - unpacking data and processing the filter. Filtering performance can be improved by replacing conditions like if ( ( i==(ACCESSOR_VALUES)m_tValue ) ^ (!EQ) ) with predication using _mm256_cmpeq_epi32 and similar intrinsics.

githubmanticore commented 1 year ago

➤ Ilya Kuznetsov commented:

We will also need to add specializations for range filters to replace the generic if ( RANGE_EVAL::Eval ( (VALUES)i, (VALUES)m_iMinValue, (VALUES)m_iMaxValue ) ) that we have now.