search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.88k stars 1.05k forks source link

[docdb] Don't needlessly set intent upper bound #3567

Open jaki opened 4 years ago

jaki commented 4 years ago

Jira Link: DB-1433 I noticed that intent upper bound for intent aware iterators gets set frequently. For example, we have

W0205 15:05:37.372021 11322 intent_aware_iterator.cc:991] ResetIntentUpperbound = x
    @     0x7fdc4c0c1536  yb::docdb::IntentAwareIterator::ResetIntentUpperbound() (src/yb/docdb/intent_aware_iterator.cc:992)
    @     0x7fdc4c0c1807  yb::docdb::IntentAwareIterator::SetIntentUpperbound() (src/yb/docdb/intent_aware_iterator.cc:981)
    @     0x7fdc4c0c6a14  yb::docdb::IntentAwareIterator::SeekIntentIterIfNeeded() (src/yb/docdb/intent_aware_iterator.cc:531)
    @     0x7fdc4c0c6e80  yb::docdb::IntentAwareIterator::valid() (src/yb/docdb/intent_aware_iterator.cc:555)
    @     0x7fdc4c0a6f62  yb::docdb::DocRowwiseIterator::HasNext() const (src/yb/docdb/doc_rowwise_iterator.cc:641)
    @     0x7fdc4c0d110b  yb::docdb::PgsqlReadOperation::Execute(yb::common::YQLStorageIf const&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, yb::ReadHybridTime const&, yb::Schema const&, yb::Schema const*, yb::PgsqlResultSet*, yb::HybridTime*) (src/yb/docdb/pgsql_operation.cc:437)

and

W0205 13:36:58.458866  4152 intent_aware_iterator.cc:991] ResetIntentUpperbound = x
    @     0x7fdc4c0c1536  yb::docdb::IntentAwareIterator::ResetIntentUpperbound() (src/yb/docdb/intent_aware_iterator.cc:992)
    @     0x7fdc4c0c658c  yb::docdb::IntentAwareIterator::PreparePrev(yb::Slice const&) (src/yb/docdb/intent_aware_iterator.cc:463)
    @     0x7fdc4c0c66c4  yb::docdb::IntentAwareIterator::PrevDocKey(yb::Slice const&) (src/yb/docdb/intent_aware_iterator.cc:489)
    @     0x7fdc4c0a426d  yb::docdb::DocRowwiseIterator::AdvanceIteratorToNextDesiredRow() const (src/yb/docdb/doc_rowwise_iterator.cc:616)
    @     0x7fdc4c0a79a0  yb::docdb::DocRowwiseIterator::HasNext() const (src/yb/docdb/doc_rowwise_iterator.cc:725)
    @     0x7fdc4c0d110b  yb::docdb::PgsqlReadOperation::Execute(yb::common::YQLStorageIf const&, std::chrono::time_point<yb::CoarseMonoClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, yb::ReadHybridTime const&, yb::Schema const&, yb::Schema const*, yb::PgsqlResultSet*, yb::HybridTime*) (src/yb/docdb/pgsql_operation.cc:437)

code paths that reset intent upper bound, and this may happen too often.

I think that this can be reduced, though I am not sure whether there would be much performance gains. One area that could probably do away with even setting an intent upper bound is backwards iterators because I don't think that they use them.

Here is a bad case in YSQL:

CREATE TABLE h (i int PRIMARY KEY);
BEGIN;
INSERT INTO h (i) SELECT * FROM generate_series(1, 100);
SELECT i FROM h ORDER BY i DESC;

The SELECT calls ResetIntentUpperbound over 100 times.

kutayzorlu commented 4 years ago

I got the same error also when a client connecting : only SELECT the table .


W0708 03:00:20.998194   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 5538f941-3203-4b2b-b312-2058a9d3302f
I0708 03:00:20.998253   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 9d10bc7f-0e81-45a9-bd11-3361933aab36, for: get commit time
W0708 03:00:20.998281   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 9d10bc7f-0e81-45a9-bd11-3361933aab36
I0708 03:00:20.998307   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 6b1166e4-52a3-4481-aad6-544d37a57c90, for: get commit time
W0708 03:00:20.998327   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 6b1166e4-52a3-4481-aad6-544d37a57c90
I0708 03:00:20.998344   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: cc517ddd-1637-49e3-98cd-c7a104285c30, for: get commit time
W0708 03:00:20.998363   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: cc517ddd-1637-49e3-98cd-c7a104285c30
I0708 03:00:20.998379   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 3f6c10db-db20-4e22-9e21-bed0c06cf2cf, for: get commit time
W0708 03:00:20.998397   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 3f6c10db-db20-4e22-9e21-bed0c06cf2cf
I0708 03:00:20.998415   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 58380bde-a89a-4c82-8afe-7225b61b24d8, for: get commit time
W0708 03:00:20.998432   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 58380bde-a89a-4c82-8afe-7225b61b24d8
I0708 03:00:20.998447   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: f6703b21-d477-4b58-b7b4-1de439a21991, for: get commit time
W0708 03:00:20.998467   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: f6703b21-d477-4b58-b7b4-1de439a21991
I0708 03:00:20.998508   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: ed2e111c-f0b9-43fb-88ff-cf73f114b35e, for: get commit time
W0708 03:00:20.998529   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: ed2e111c-f0b9-43fb-88ff-cf73f114b35e
I0708 03:00:20.998545   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 230f2986-f9cd-4e97-875a-54e2ef663b15, for: get commit time
W0708 03:00:20.998564   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 230f2986-f9cd-4e97-875a-54e2ef663b15
I0708 03:00:20.998581   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 8f90d92f-498e-4ffc-8d9e-b750d49a9e9d, for: get commit time
W0708 03:00:20.998600   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 8f90d92f-498e-4ffc-8d9e-b750d49a9e9d
I0708 03:00:20.998616   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: e3093a0a-9743-4551-8e24-f06b33c071ed, for: get commit time
W0708 03:00:20.998633   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: e3093a0a-9743-4551-8e24-f06b33c071ed
I0708 03:00:20.998762   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 66ab2417-7389-46ae-99fb-aaa8ad2ba315, for: get commit time
W0708 03:00:20.998811   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 66ab2417-7389-46ae-99fb-aaa8ad2ba315
I0708 03:00:20.998831   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 85dc4cf3-bf15-4fb7-b8c4-80e0189af3a0, for: get commit time
W0708 03:00:20.998850   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 85dc4cf3-bf15-4fb7-b8c4-80e0189af3a0
I0708 03:00:20.998869   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: 0a7e12fb-4f11-4214-b2df-d987fbba6094, for: get commit time
W0708 03:00:20.998901   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: 0a7e12fb-4f11-4214-b2df-d987fbba6094
I0708 03:00:20.998921   446 transaction_participant.cc:999] T a70462a6810f4c80ae61622502ce7c05 P 20214e0900834640988f4f09f437302d: Transaction not found: a73f2102-8085-479a-b586-303403aa0377, for: get commit time
W0708 03:00:20.998944   446 intent_aware_iterator.cc:142] Intent for transaction w/o metadata: a73f2102-8085-479a-b586-303403aa0377