tarantool / crud

Easy assess to data stored in vshard cluster
BSD 2-Clause "Simplified" License
40 stars 14 forks source link

scan: fix filter erroneous early exit #419

Closed DifferentialOrange closed 4 months ago

DifferentialOrange commented 5 months ago

The issue described below is related to the read operations which allows to scan: crud.select, crud.pairs, crud.count, readview:select and readview:pairs.

The erroneous behavior reported by [1] and #418 is as follows:

The reason is as follows. Scanning read operates with two entities: an iterator and a filter. The iterator includes an index, a starting value and iterator type (EQ, GT, etc.). The iterator is built from conditions, if possible, otherwise primary index is used. Remaining conditions form the filter, so the actual result satisfies all operation conditions.

The filter supports early exit. Let's consider the following example. For crud.select(space, {{'>=', 'id', 1}, {'<=', 'id', 10}}), where id is an index (or an indexed field), the iterator uses index id, starts from key = 1 and goes by GE rules, covering [1, +inf) ordered keys. On the other hand, when iterator reaches the tuple with id = 11, all tuples after this one will never satisfy the second condition, because our iterator yields tuples sorted by id (due to underlying index). So filter tells than there is no reason to scan anymore, and we finish the scanning procedure.

Before this patch, the function behind early exit decision had worked as follows: "if the condition is an index, we go in forward (reverse) order and <= or < (>= or >) condition is violated, there is no reason to scan anymore". But the valid approach is "if the condition is SCANNING index...". Before this patch, filter had assumed that if the condition for index is specified, tuples are ordered, but it works only if iterator uses the same index as in the condition. This patch fixes the issue.

The erroneous behavior may happen in the following case:

  1. https://jira.vk.team/browse/TNT-941

Closes #418