tarantool / tarantool

Get your data in RAM. Get compute close to data. Enjoy the performance.
https://www.tarantool.io
Other
3.42k stars 380 forks source link

vinyl: skip invisible read sources #10851

Closed locker closed 2 days ago

locker commented 6 days ago

A Vinyl read iterator scans all read sources (memory and disk levels) even if it's executed in a read view from which most of the sources are invisible. As a result, a long running scanning request may spend most of the time skipping invisible statements. The situation is exacerbated if the instance is experiencing a heavy write load because it would pile up old statement versions in memory and force the iterator to skip over them after each disk read.

Since the replica join procedure in Vinyl uses a read view iterator under the hood, the issue is responsible for a severe performance degradation of the master instance and the overall join procedure slowdown when a new replica is joined to an instance running under a heavy write load.

Let's fix this issue by making a read iterator skip read sources that aren't visible from its read view.

Closes #10846

coveralls commented 6 days ago

Coverage Status

coverage: 87.335% (+0.02%) from 87.315% when pulling 9c4bc38033eed97eea6d2de69ff8e3a937ab944d on locker:vy-skip-invisible-read-src into 017700dccdb04edd8470af620c69f3e7c572895b on tarantool:master.

locker commented 2 days ago

Cherry-picked to 2.11 and 3.2.