Phoenix-Protocol-Group / phoenix-contracts

Source code of the smart contracts of the Phoenix DeFi hub DEX protocol
GNU General Public License v3.0
9 stars 5 forks source link

Pool: Swap performance metering #165

Open ueco-jb opened 8 months ago

ueco-jb commented 8 months ago

We need to measure how much memory does a swap operation takes, in couple variants of the pools and parameters. Research how to perform metering, I have some small doc from soroban documentation: https://soroban.stellar.org/docs/fundamentals-and-concepts/fees-and-metering

We need to measure memory usage of swap and how it compares with chained swaps in multihop.

gangov commented 8 months ago

so after some digging turns out that we can estimate the CPU and memory usage during our calls with the following piece of code at the end of our tests: env.budget().print();

I did this on one of the tests that simulate multihop swap between 3 pools and the results are way of the charts:

running 1 test
=======================================================
Cpu limit: 18446744073709551615; used: 539928523
Mem limit: 18446744073709551615; used: 83713876
=======================================================
CostType                 cpu_insns      mem_bytes      
WasmInsnExec             1506498        0              
WasmMemAlloc             0              41222144       
HostMemAlloc             9479263        2877977        
HostMemCpy               916490         0              
HostMemCmp               1033995        0              
DispatchHostFunction     177262         0              
VisitObject              588276         0              
ValSer                   406268         34194          
ValDeser                 4492           240            
ComputeSha256Hash        9551895        2240           
ComputeEd25519PubKey     0              0              
MapEntry                 1929942        0              
VecEntry                 0              0              
VerifyEd25519Sig         0              0              
VmMemRead                70838          0              
VmMemWrite               4017           0              
VmInstantiation          514230037      39576717       
VmCachedInstantiation    0              0              
InvokeVmFunction         29250          364            
ComputeKeccak256Hash     0              0              
ComputeEcdsaSecp256k1Key 0              0              
ComputeEcdsaSecp256k1Sig 0              0              
RecoverEcdsaSecp256k1Key 0              0              
Int256AddSub             0              0              
Int256Mul                0              0              
Int256Div                0              0              
Int256Pow                0              0              
Int256Shift              0              0              
=======================================================

test tests::swap::swap_three_equal_pools_no_fees ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 19 filtered out; finished in 0.29s

[Process exited 0]

Even just a single pool swap from multihop costs us a lot right now:

running 1 test
=======================================================
Cpu limit: 18446744073709551615; used: 200647414
Mem limit: 18446744073709551615; used: 32461452
=======================================================
CostType                 cpu_insns      mem_bytes      
WasmInsnExec             540234         0              
WasmMemAlloc             0              16711680       
HostMemAlloc             3964492        988031         
HostMemCpy               345684         0              
HostMemCmp               325132         0              
DispatchHostFunction     65224          0              
VisitObject              230148         0              
ValSer                   181972         15402          
ValDeser                 2246           120            
ComputeSha256Hash        3729702        1120           
ComputeEd25519PubKey     0              0              
MapEntry                 659002         0              
VecEntry                 0              0              
VerifyEd25519Sig         0              0              
VmMemRead                25698          0              
VmMemWrite               1339           0              
VmInstantiation          190565291      14744959       
VmCachedInstantiation    0              0              
InvokeVmFunction         11250          140            
ComputeKeccak256Hash     0              0              
ComputeEcdsaSecp256k1Key 0              0              
ComputeEcdsaSecp256k1Sig 0              0              
RecoverEcdsaSecp256k1Key 0              0              
Int256AddSub             0              0              
Int256Mul                0              0              
Int256Div                0              0              
Int256Pow                0              0              
Int256Shift              0              0              
=======================================================

test tests::swap::swap_single_pool_no_fees ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 19 filtered out;
 finished in 0.11s

[Process exited 0]

The following are the results from running a simple swap inside the pool contract tests:

running 1 test
=======================================================
Cpu limit: 18446744073709551615; used: 73556781
Mem limit: 18446744073709551615; used: 11320218
=======================================================
CostType                 cpu_insns      mem_bytes      
WasmInsnExec             73050          0              
WasmMemAlloc             0              5570560        
HostMemAlloc             4180789        643981         
HostMemCpy               297186         0              
HostMemCmp               248815         0              
DispatchHostFunction     7627           0              
VisitObject              262224         0              
ValSer                   153871         12696          
ValDeser                 2246           120            
ComputeSha256Hash        1825023        720            
ComputeEd25519PubKey     0              0              
MapEntry                 536731         0              
VecEntry                 0              0              
VerifyEd25519Sig         0              0              
VmMemRead                3875           0              
VmMemWrite               0              0              
VmInstantiation          65961969       5092099        
VmCachedInstantiation    0              0              
InvokeVmFunction         3375           42             
ComputeKeccak256Hash     0              0              
ComputeEcdsaSecp256k1Key 0              0              
ComputeEcdsaSecp256k1Sig 0              0              
RecoverEcdsaSecp256k1Key 0              0              
Int256AddSub             0              0              
Int256Mul                0              0              
Int256Div                0              0              
Int256Pow                0              0              
Int256Shift              0              0              
=======================================================

test tests::swap::simple_swap ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out;
 finished in 0.06s

[Process exited 0]

The Cpu and Mem limits are so high because we set it to unlimited at the beginning of our test via env.budget().reset_unlimited(); but we can see that the actual usage is off the charts:

Multihop 3 pool swap:

Cpu used: 539_928_523
Mem used: 83_713_876

Multihop single pool swap:

Cpu used: 200_647_414
Mem used: 32_461_452

Liquidity pool simple swap:

Cpu used: 73_556_781
Mem used: 11_320_218

having Testnet limits of Cpu: 100_000_000 and Memory: 40MB

Another point of interest would be

CostType                 cpu_insns        mem_bytes
VmInstantiation          514230037      39576717       

One step in the right direction is to be using soroban optimize --wasm cli command and then calling this optimized wasm file in the contracts. An example would be something like:

pub mod factory {
    soroban_sdk::contractimport!(
---        file = "../../target/wasm32-unknown-unknown/release/phoenix_factory.wasm"
    );
}
pub mod factory {
    soroban_sdk::contractimport!(
+++       file = "../../target/wasm32-unknown-unknown/release/phoenix_factory.optimized.wasm"
    );
}

We can probably use that in a separate CI/CD pipeline only when deploying to Testnet and not for the feature branches

gangov commented 8 months ago

interesting addition https://github.com/stellar/rs-soroban-sdk/issues/1113

gangov commented 8 months ago

to summarize the story so far:

The memory usage in the budget represents the modeled RAM usage, which is not the same as the storage usage. 
Our data is stored in memory before being flushed to the ledger, but RAM usage is not a good indicator of storage usage. 
We are interested in the amount of I/O our contracts use, but Soroban unfortunately does not have a any tools for that yet.

As an alternative we can try to simulate our contract invocations and see how much resources it consumes.