search

hamedpro / freeflow-core

🩵⚡️next generation of decentralised collaboration
3 stars 1 forks source link

cpu bottleneck in new transaction #238

Open hamedpro opened 1 year ago

hamedpro commented 1 year ago

where there is a lot of transactions there, submitting a new transaction takes too long. even maybe 30 seconds!!! i'm sure somewhere there's a heavy computation under a loop. something like computing a cache or something in each iteration of a long loop. measure time of each task in new tr func and find that

hamedpro commented 1 year ago

almost all of the problem was found out to be custom_find_unique func which is used in sync_websocket_clients. its really time consuming. there is a find_duplicate_pairs function inside it which has 2 parts and each of them took under 15 ms but when this number gets multiplied by a large number, result will be huge. (around a second in our case with 305 transactions)

hamedpro commented 1 year ago

added a new algo for when custom compare function is equal to === . in our case time was decreased from around 900 ms to less than a 1 ms !!!!! just now that commit will close this issue

hamedpro commented 1 year ago

still network panel shows long waiting for server response for some of /new_transaction requests. check it and close the issue

hamedpro commented 1 year ago

calc_user_discoverable_transactions in core takes up to 380 ms and that is probably because of mapping an array to thing_transactions. when even 2-3 ms is multiplied by for example 200 it will be 600 ms.