search

helium / rosetta-helium

Rosetta implementation for helium
18 stars 4 forks source link

Slow API response #53

Open GeekArthur opened 2 years ago

GeekArthur commented 2 years ago

Originally the slow API response issue was detected by @Arslan2 with the following description:

The response time for rosetta APIs are very high. Sometimes it takes around 20+ seconds to get response from /network/status particularly. At one point /block API took ~2 minutes to return the response. You can see the below logs for response time. The instance we are using is c5.2xlarge

Response time log:

2021-12-21T02:24:05.813Z info:  Starting mainnet server
2021-12-21T02:24:05.901Z info:  Express server listening on port 3000
2021/12/21 02:24:09 POST /network/list 406.545µs
2021/12/21 02:24:23 POST /network/status 13.944491701s
2021/12/21 02:24:23 POST /network/options 373.621µs
2021/12/21 02:24:23 POST /network/list 224.608µs
2021/12/21 02:24:48 POST /network/status 25.141337733s
2021/12/21 02:24:49 POST /network/status 24.513635085s
2021/12/21 02:24:49 POST /network/options 175.902µs
2021/12/21 02:24:59 POST /network/status 10.746308049s
2021/12/21 02:25:08 POST /network/status 9.277194867s
2021/12/21 02:25:08 POST /network/status 8.823293569s
2021/12/21 02:25:17 POST /block 8.325979479s
2021/12/21 02:25:18 POST /block 9.298215533s
2021/12/21 02:25:20 POST /block 11.566144474s
2021/12/21 02:25:20 POST /block 12.032916917s
2021/12/21 02:25:24 POST /account/balance 3.545710312s
2021/12/21 02:25:24 POST /account/balance 3.564074655s
2021/12/21 02:25:24 POST /account/balance 3.569971797s
2021/12/21 02:25:24 POST /account/balance 3.575337309s
2021/12/21 02:25:24 POST /account/balance 3.577154117s
2021/12/21 02:25:24 POST /account/balance 3.583850659s
2021/12/21 02:25:24 POST /account/balance 3.58459173s
2021/12/21 02:25:24 POST /account/balance 3.584181568s
2021/12/21 02:25:24 POST /account/balance 56.198256ms
2021/12/21 02:25:27 POST /account/balance 3.521343305s
2021/12/21 02:25:27 POST /account/balance 3.51512191s
2021/12/21 02:25:27 POST /account/balance 3.515279885s
2021/12/21 02:25:27 POST /account/balance 3.506439365s
2021/12/21 02:25:27 POST /account/balance 3.514701696s
2021/12/21 02:25:27 POST /account/balance 3.511495163s
2021/12/21 02:25:27 POST /account/balance 3.58150446s
2021/12/21 02:25:28 POST /block 11.130667707s
2021/12/21 02:25:30 POST /block 12.072155562s
2021/12/21 02:25:31 POST /block 10.553223571s
2021/12/21 02:25:32 POST /network/status 23.376350922s
2021/12/21 02:25:38 POST /network/status 18.832427751s
2021/12/21 02:25:43 POST /network/status 10.999179087s
2021/12/21 02:25:50 POST /network/status 12.648286707s
2021/12/21 02:25:58 POST /network/status 14.836957747s
2021/12/21 02:26:05 POST /network/status 14.691739717s
2021/12/21 02:26:18 POST /network/status 19.877239957s
2021/12/21 02:26:31 POST /network/status 26.356417822s
2021/12/21 02:26:36 POST /network/status 28.456843596s
2021/12/21 02:26:40 POST /network/status 22.512911457s
2021/12/21 02:26:44 POST /network/status 13.232532074s
2021/12/21 02:26:48 POST /network/status 7.580154111s
2021/12/21 02:26:52 POST /network/status 7.360908066s
2021/12/21 02:26:59 POST /network/status 10.20306974s
2021/12/21 02:27:02 POST /network/status 10.468603016s
2021/12/21 02:27:09 POST /network/status 10.196797248s
2021/12/21 02:27:13 POST /network/status 10.51673449s
2021/12/21 02:27:20 POST /network/status 10.689286084s
2021/12/21 02:27:24 POST /network/status 11.627868144s
2021/12/21 02:27:31 POST /network/status 10.766856335s
2021/12/21 02:27:34 POST /network/status 9.875630769s
2021/12/21 02:27:41 POST /network/status 9.970917721s

@syuan100 Already did some optimizations, we need to test against the latest version to see if the performance issue still exists

syuan100 commented 2 years ago

This may be related to #51

We might just take the sync_status parameter out for now and see if that improves things for that endpoint.

The /block endpoint will naturally take longer due to the 100-200k ops required to be parsed for the reward blocks. I'll see if there are any optimizations we can do to parse it faster.

GeekArthur commented 2 years ago

@Arslan2 can still hit the same issue on multiple endpoints

/network list/status/options/balance APIs are now taking under ~2 seconds on AWS for mainnet

Can we address those performance issues if the specific endpoint response time does not naturally take that long?