search

ubiquity-os-marketplace / command-wallet

Commands related to wallet update and query.
0 stars 9 forks source link

Use organization package `rpc-handler` to get the fastest provider available #3

Closed gentlementlegen closed 2 months ago

gentlementlegen commented 4 months ago

I had no luck when trying to use the rpc package so I want to do it in a separate PR. This endpoint was quite reliable during my tests which is why I kept it. You can even try it in my testing repo if you want.

_Originally posted by @gentlementlegen in https://github.com/ubiquibot/command-wallet/pull/2#discussion_r1663135402_

Depends on https://github.com/ubiquity/rpc-handler/pull/45 to be published.

gentlementlegen commented 3 months ago

The following issue would need to be resolved to avoid the bugs that I encountered: https://github.com/ubiquity/rpc-handler/issues/32

ubiquibot[bot] commented 3 months ago

@gentlementlegen the deadline is at 2024-08-08T04:27:09.489Z

gentlementlegen commented 3 months ago

The main issue currently is that the rpc-handler package makes too many requests and triggers a cancellation on the worker. There should be an option to limit the amount of tested networks in the package.

gentlementlegen commented 3 months ago

Update: I've been working all day on this, by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs. Every time I get the following message:

"HttpError: Too many subrequests."

Maybe our package is not suited for this. In the end we do not care about having the fastest RPC but any that works. Maybe it would be best to simply use the same package we use in rpc-handler, and try them one by one until the first one that works to avoid bursting the request limit within the workers. @rndquu thoughts?

rndquu commented 3 months ago

Update: I've been working all day on this, by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs. Every time I get the following message:

"HttpError: Too many subrequests."

Maybe our package is not suited for this. In the end we do not care about having the fastest RPC but any that works. Maybe it would be best to simply use the same package we use in rpc-handler, and try them one by one until the first one that works to avoid bursting the request limit within the workers. @rndquu thoughts?

You mean the HttpError: Too many subrequests error is thrown by Cloudflare while using the rpc-handler package, right?

gentlementlegen commented 3 months ago

@rndquu That is correct. The count for total request doesn't care if the request got aborted or cancelled it seems, so we always reach the limit by using out package.

rndquu commented 3 months ago

@gentlementlegen @Keyrxng

So the issue is that https://github.com/ubiquity/rpc-handler can't be used in a cloudflare environment (at least for now) because of a great number of HTTP requests in the background.

Perhaps we should add a new method to https://github.com/ubiquity/rpc-handler like handler.getAnyRpcProvider() which would simply return the 1st item from https://chainlist.org/chain/1 without any background checks?

Update: I've just double checked and there's already an issue for that https://github.com/ubiquity/rpc-handler/issues/43 so it makes sense to put current issue on hold and implement https://github.com/ubiquity/rpc-handler/issues/43 first.

Keyrxng commented 3 months ago

Yeah I think we could create a generator function that tests the network RPCs one by one and we yield the first to pass with checks if possible or none if we still hit that error

@gentlementlegen have you tried passing a hex network id and manually assigning network and runtime rpcs? Hacky and might not work but I'm sure the Networkish type also includes hex but rpc-handler wouldn't find a list of networkd RPCs.

I quickly read through the code and I think that you always end up with all of the endpoints instead of just the custom ones you passed?

by changing the rpc-handler package to either include only specified URLs or even hardcoding a list of URLs.

Oh I think you meant rewriting it, my mistake. Although I think the above still applies but what might stop us is if the random network RPC that we select is a slow/faulty one then it'll likely hit the max request time. So like @rndquu suggests just serving a url from the package without checks but that's the same as hardcoding 3 or 4 known good ones and shuffling isn't it?

gentlementlegen commented 3 months ago

I was trying to solve the issue on the rpc package by skipping all the other endpoints but I didn't expect them to still count as network calls, even canceled or failed. A method to try them one by one seems to be a reliable idea, I will look into that.

ubiquibot[bot] commented 3 months ago

@gentlementlegen the deadline is at 2024-08-18T14:08:53.387Z

ubiquity-os[bot] commented 2 months ago

[ 80.92 WXDAI ]

@gentlementlegen
Contributions Overview
View Contribution Count Reward
Issue Task 1 25
Issue Specification 1 17.7
Issue Comment 5 38.22
Review Comment 2 0
Conversation Incentives
Comment Formatting Relevance Reward
I had no luck when trying to use the rpc package so I want to do…
17.7 
content:
  p:
    count: 59
    score: 1
  em:
    count: 6
    score: 0
wordValue: 0.1
formattingMultiplier: 3
1 17.7
The following issue would need to be resolved to avoid the bugs …
3.2 
content:
  p:
    count: 16
    score: 1
wordValue: 0.2
formattingMultiplier: 1
0.6 1.92
The main issue currently is that the `rpc-handler` packa…
7.2 
content:
  p:
    count: 35
    score: 1
  code:
    count: 1
    score: 1
wordValue: 0.2
formattingMultiplier: 1
0.8 5.76
Update: I've been working all day on this, by changing the `…
20.2 
content:
  p:
    count: 95
    score: 1
  code:
    count: 6
    score: 1
  pre:
    count: 4
    score: 0
wordValue: 0.2
formattingMultiplier: 1
0.9 18.18
@rndquu That is correct. The count for total request doesn't car…
6 
content:
  p:
    count: 30
    score: 1
wordValue: 0.2
formattingMultiplier: 1
0.7 4.2
I was trying to solve the issue on the rpc package by skipping a…
10.2 
content:
  p:
    count: 51
    score: 1
wordValue: 0.2
formattingMultiplier: 1
0.8 8.16
Resolves #3 Depends on https://github.com/ubiquity/rpc-handler…
0 
content:
  p:
    count: 6
    score: 1
  ul:
    count: 18
    score: 0
  li:
    count: 18
    score: 1
wordValue: 0
formattingMultiplier: 0
0.3 -
The changelog is automatically generated by `release-please&…
0 
content:
  p:
    count: 39
    score: 1
  code:
    count: 2
    score: 1
wordValue: 0.2
formattingMultiplier: 0
1 -

[ 2.395 WXDAI ]

@rndquu
Contributions Overview
View Contribution Count Reward
Issue Comment 2 2.395
Conversation Incentives
Comment Formatting Relevance Reward
You mean the `HttpError: Too many subrequests` error is …
0.575 
content:
  p:
    count: 18
    score: 1
  code:
    count: 5
    score: 1
wordValue: 0.1
formattingMultiplier: 0.25
0.6 0.345
@gentlementlegen @Keyrxng So the issue is that https://github.…
2.05 
content:
  p:
    count: 81
    score: 1
  code:
    count: 1
    score: 1
wordValue: 0.1
formattingMultiplier: 0.25
1 2.05

[ 12.25 WXDAI ]

@Keyrxng
Contributions Overview
View Contribution Count Reward
Issue Comment 1 12.25
Conversation Incentives
Comment Formatting Relevance Reward
Yeah I think we could create a generator function that tests the…
17.5 
content:
  p:
    count: 173
    score: 1
  code:
    count: 2
    score: 1
wordValue: 0.1
formattingMultiplier: 1
0.7 12.25

[ 3.5 WXDAI ]

@0x4007
Contributions Overview
View Contribution Count Reward
Review Comment 2 3.5
Conversation Incentives
Comment Formatting Relevance Reward
Nice to have, but seems tedious for other people to ensure this …
2.7 
content:
  p:
    count: 27
    score: 1
wordValue: 0.1
formattingMultiplier: 1
1 2.7
```suggestion ``` Really doesn't matter …
0.8 
content:
  pre:
    count: 1
    score: 0
  code:
    count: 1
    score: 1
  p:
    count: 7
    score: 1
wordValue: 0.1
formattingMultiplier: 1
1 0.8