permaweb / ao

The ao component and tools Monorepo - 🐰 🕳️ 👈
https://cookbook_ao.arweave.dev
Other
168 stars 55 forks source link

perf(su): increase the speed of building bundles, or move it into the background #877

Open VinceJuliano opened 1 month ago

VinceJuliano commented 1 month ago

Background

Benchmarks have been reported that building the bundle is slowing down the write process

2024-07-10T20:28:21Z INFO su::domain::logger] === CREATING MESSAGE - 33.889ms [2024-07-10T20:28:21Z INFO su::domain::logger] saved message [2024-07-10T20:28:21Z INFO su::domain::logger] === SAVING MESSAGE - 3.887ms

Solution

Look into speeding up the building of the bundles, builder.rs/bytes.rs. Or if that is not possible, explore returning to the user and then doing all the building and saving in the background, after validating incoming DataItem.

ppedziwiatr commented 1 month ago

Hey, as I've mentioned - the most time here (i.e. in the CREATING MESSAGE part) is used on signing the data-items..the assignment data item and the nested bundle that contains both the assignment and original message - e.g.:

[2024-07-10T20:50:27Z INFO  su::domain::logger] === SIGNING ASSIGNMENT - 19.673ms
[2024-07-10T21:01:16Z INFO  su::domain::logger] === SIGNING NESTED BUNDLE - 13.124ms

It would be best if the nested bundle was created by a separate, background process (that would be also responsible for sending it to Turbo/Irys and - ideally - verififying after some blocks (50? 100?) whether the data items are indeed on Arweave).

Not sure how to deal with the signature of the assignment - but it would be great if it could be taken out of the locking part of code. Removing signing from the locked part of code (https://github.com/permaweb/ao/blob/main/servers/su/src/domain/core/flows.rs#L175-L188) would significantlly speed up the messages registration in the SU.

In general - the whole process of registering a new message in the SU takes usually (on my M2 Pro) ~50ms (obviously on a release build). So shaving off over 30ms "just" by removing the signing of the data items would be a great win here.