search

waku-org / nwaku

Waku node and protocol.
Other
199 stars 51 forks source link

chore: store queries for content topics take longer with cursor #2963

Open stubbsta opened 1 month ago

stubbsta commented 1 month ago

Background

When making a storev3 "get messages" request with arbitrary query criteria and no cursor, the response is received much faster than when a cursor is included in the query

Details

On average the query duration is at least twice as long for queries that include a cursor. Here is an example of the query parameters:

"{'peerAddr': {'/dns4/store-02.do-ams3.status.prod.status.im/tcp/30303/p2p/16Uiu2HAm9aDJPkhGxc2SFcEACTFdZ91Q5TJjp76qZEhq9iF59x7R'}, 'pubsubTopic': {'/waku/2/rs/16/32'}, 'contentTopics': {'/waku/1/0x0ba4cd54/rfc26,/waku/1/0x2cd89333/rfc26,/waku/1/0x1d87a3cd/rfc26,/waku/1/0xf52f9686/rfc26,/waku/1/0x7a02f98c/rfc26,/waku/1/0x185bbe63/rfc26,/waku/1/0x3263a581/rfc26,/waku/1/0x07a60173/rfc26,/waku/1/0xbc0a6d4a/rfc26,/waku/1/0x0de4b636/rfc26,/waku/1/0x2f1b19f4/rfc26,/waku/1/0xa2987838/rfc26,/waku/1/0x90da9ebc/rfc26,/waku/1/0x273c8b02/rfc26,/waku/1/0xb74df338/rfc26,/waku/1/0xdced1732/rfc26,/waku/1/0x98919cb7/rfc26,/waku/1/0x40b24197/rfc26,/waku/1/0xffdd153c/rfc26,/waku/1/0x7aaf28e7/rfc26,/waku/1/0x6cd1c858/rfc26,/waku/1/0x44b029de/rfc26,/waku/1/0xb7b52398/rfc26,/waku/1/0x44db1d5e/rfc26,/waku/1/0x42c4e775/rfc26,/waku/1/0xd586b26d/rfc26,/waku/1/0xc4eeaf67/rfc26,/waku/1/0x5f720d24/rfc26,/waku/1/0xc804e40c/rfc26,/waku/1/0x14a93183/rfc26,/waku/1/0x692b412a/rfc26,/waku/1/0x236da80c/rfc26,/waku/1/0x4612b875/rfc26,/waku/1/0x4696ddf1/rfc26,/waku/1/0x9da54487/rfc26,/waku/1/0xd19ef198/rfc26,/waku/1/0xab5ffc85/rfc26,/waku/1/0x678be1ba/rfc26,/waku/1/0x68a5ccb2/rfc26,/waku/1/0xec639454/rfc26,/waku/1/0xe550dd44/rfc26,/waku/1/0xaff53976/rfc26,/waku/1/0xce498efb/rfc26,/waku/1/0x9bef037f/rfc26,/waku/1/0x019bcad8/rfc26,/waku/1/0x145bfd72/rfc26,/waku/1/0x538017c9/rfc26,/waku/1/0x4441abc9/rfc26,/waku/1/0x8de8f186/rfc26,/waku/1/0x929d3bc7/rfc26,/waku/1/0x9ba6609e/rfc26,/waku/1/0x55af6adb/rfc26,/waku/1/0xb5ddd12d/rfc26,/waku/1/0x91ddcfb6/rfc26,/waku/1/0xf3a9da50/rfc26,/waku/1/0x0080803f/rfc26,/waku/1/0x5fc435cb/rfc26,/waku/1/0x7b9dcfa5/rfc26,/waku/1/0x7d281fed/rfc26,/waku/1/0xd53f2af7/rfc26,/waku/1/0xe5f61b62/rfc26,/waku/1/0xdea01974/rfc26,/waku/1/0x0ccbb8ca/rfc26,/waku/1/0x8f19c9a2/rfc26,/waku/1/0xc08e94aa/rfc26,/waku/1/0xb6368e8a/rfc26,/waku/1/0x98de75de/rfc26,/waku/1/0xa0f174f3/rfc26,/waku/1/0x8bddaef2/rfc26,/waku/1/0xf13176e3/rfc26,/waku/1/0x1a381c17/rfc26,/waku/1/0x2fe862ca/rfc26,/waku/1/0xcc4f7f94/rfc26,/waku/1/0xfbe1741a/rfc26,/waku/1/0x3165779f/rfc26,/waku/1/0x2f97a551/rfc26,/waku/1/0xa133d621/rfc26,/waku/1/0x5c32bfbd/rfc26,/waku/1/0xfddf2992/rfc26,/waku/1/0x5e4d9b31/rfc26,/waku/1/0xa9f7eb66/rfc26,/waku/1/0x410c2bff/rfc26,/waku/1/0x7c9e67ab/rfc26,/waku/1/0x239ee8d3/rfc26,/waku/1/0x062dba0f/rfc26,/waku/1/0xfe9137ff/rfc26,/waku/1/0xa5130468/rfc26,/waku/1/0xd774ad51/rfc26,/waku/1/0xbdf8a655/rfc26,/waku/1/0x8714d0fa/rfc26,/waku/1/0xde88507c/rfc26,/waku/1/0x5f6cd757/rfc26,/waku/1/0x6f27e4b5/rfc26,/waku/1/0x50143bfc/rfc26,/waku/1/0x4b6ca0f6/rfc26,/waku/1/0xa4d9d91b/rfc26,/waku/1/0x42f4395f/rfc26,/waku/1/0x5b10759e/rfc26,/waku/1/0x1041bf52/rfc26,/waku/1/0xf5a7c078/rfc26'}, 'startTime': {1723068000000000000}, 'endTime': {1723111200000000000}, 'ascending': 'true', 'includeData': 'false', 'pageSize': 100, 'cursor': None}"

and the same can be used with this cursor: SFbykRpYfEnZfDxJdfb1z90OjHEaYWMI4FtpD/WzdS8=

Acceptance criteria

Investigate why the query with a cursor takes so much longer and see if it can be improved

stubbsta commented 1 month ago

The same behaviour is observed when only querying for 10 content topics

Ivansete-status commented 2 weeks ago

Something interesting is that after applying ANALYZE messages the performance got better in status.prod ams db.

On the other hand, I don't see a clear evidence of being slower the queries in the cursor than the first query. For example, while checking with the message-finder app:

image