status-im / status-go

The Status module that consumes go-ethereum
https://status.im
Mozilla Public License 2.0
728 stars 247 forks source link

fix(WC)_: Create indexes for WC queries #5989

Open alexjba opened 1 month ago

alexjba commented 1 month ago

Create the indexes needed for WalletConnect queries based on .expert recommendations.

Fixing: https://github.com/status-im/status-desktop/issues/14975

status-im-auto commented 1 month ago

Jenkins Builds

:grey_question: Commit :hash: Finished (UTC) Duration Platform Result
:heavy_check_mark: d128fad8 #1 2024-10-24 14:18:21 ~4 min linux :package:zip
:heavy_check_mark: d128fad8 #1 2024-10-24 14:18:26 ~4 min ios :package:zip
:heavy_check_mark: d128fad8 #1 2024-10-24 14:19:35 ~6 min android :package:aar
:heavy_check_mark: d128fad8 #1 2024-10-24 14:20:06 ~6 min tests-rpc :page_facing_up:log
:heavy_multiplication_x: d128fad8 #1 2024-10-24 14:47:51 ~34 min tests :page_facing_up:log
:heavy_check_mark: d128fad8 #2 2024-10-25 06:57:17 ~30 min tests :page_facing_up:log
codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.48%. Comparing base (c72f491) to head (d128fad).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #5989 +/- ## =========================================== - Coverage 47.49% 47.48% -0.02% =========================================== Files 849 849 Lines 138606 138606 =========================================== - Hits 65836 65813 -23 - Misses 64984 64996 +12 - Partials 7786 7797 +11 ``` | [Flag](https://app.codecov.io/gh/status-im/status-go/pull/5989/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=status-im) | Coverage Δ | | |---|---|---| | [functional](https://app.codecov.io/gh/status-im/status-go/pull/5989/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=status-im) | `10.20% <ø> (+0.01%)` | :arrow_up: | | [unit](https://app.codecov.io/gh/status-im/status-go/pull/5989/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=status-im) | `46.97% <ø> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=status-im#carryforward-flags-in-the-pull-request-comment) to find out more. [see 24 files with indirect coverage changes](https://app.codecov.io/gh/status-im/status-go/pull/5989/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=status-im)
alexjba commented 4 weeks ago

@alexjba do we really need this indexing? Wondering now, how many rows do we expect from an average user? Do we do any table cleaning once the session expires?

We don't do any cleaning! And then it could become an issue if the query is not optimized. Moreover, it adds some weight to this decision when we consider that everything is processed synchronously.

Now the reasoning why we never delete sessions is another topic that should be discussed soon. What I've found so far is that this was an informed decision.

The sessions requests from dApps are persisted for offline dApps display and to have history later on when implemented in UX. https://www.notion.so/Wallet-Connect-Desktop-handover-73a52b2de7cf45b3af2905dfe164cd76?pvs=4#600332b3cce94e92815e4e4173f7aea6

saledjenic commented 4 weeks ago

But the session has its own expiration date, I don't think that we need to keep an expired session in db, but agree we should discuss that.

alexjba commented 1 week ago

@clauxx The mobile WC integration is using the sessions and dapps stored in the DB?

I see little value for these tables here, but maybe I'm missing something.

IMO these tables are useful only to show some dapps if the user starts the app and the network is down. But this doesn't mean the dapp list is valid. Once WC is connected then the dapp list can change. So we could just inform the user that the network is down and wait for the connection before showing the connected dApps.

As @saledjenic pointed out, there's this issue where we never delete sessions. So everything just piles up in the DB. From this perspective I think the DB needs to be optimized if these tables are useful to us. Either to delete sessions periodically, or to optimize the DB queries (or maybe both)

clauxx commented 1 week ago

@clauxx The mobile WC integration is using the sessions and dapps stored in the DB?

I see little value for these tables here, but maybe I'm missing something.

IMO these tables are useful only to show some dapps if the user starts the app and the network is down. But this doesn't mean the dapp list is valid. Once WC is connected then the dapp list can change. So we could just inform the user that the network is down and wait for the connection before showing the connected dApps.

As @saledjenic pointed out, there's this issue where we never delete sessions. So everything just piles up in the DB. From this perspective I think the DB needs to be optimized if these tables are useful to us. Either to delete sessions periodically, or to optimize the DB queries (or maybe both)

Aside from the offline case, we don't clear expired sessions because we were planning to show the user their expired sessions next to the active ones (also we had some requirements to show all the sessions the user had with a specific dapp).

No issues with indexing from my side.

saledjenic commented 1 week ago

also we had some requirements to show all the sessions the user had with a specific dapp

@clauxx where do those requirements come from? Does any other wallet offer you a list of expired or non-active sessions? The active session only matters nothing else.

Imagine how many entries can be here for a really active user in a year, and why, I don't see a need for that.

clauxx commented 1 week ago

also we had some requirements to show all the sessions the user had with a specific dapp

@clauxx where do those requirements come from? Does any other wallet offer you a list of expired or non-active sessions? The active session only matters nothing else.

Imagine how many entries can be here for a really active user in a year, and why, I don't see a need for that.

Went through the designs and can't find them anymore, maybe misremembering. I guess it makes sense to clean up expired sessions, cause the validity seems to be ~ a week and with heavy usage might indeed grow too much.

The active session only matters nothing else

Given the number of dapps available, it might be useful to show the user expired dapps they used before, so they can potentially re-connect to them, but it seems we don't have this requirement/user-story written anywhere, so could be irrelevant for now.