Patrik-Stas
commented
10 months ago Hi @jleach from my experience and benchmarking it's very likely that outdated genesis file is like the cause of the slowness you observe.
When we switched in aries-vcx from using indysdk based ledger client to indy-vdr, we noticed that indy-vdr implementation is more sensitive to outdated genesis files. Especially when the genesis file is faulty in a manner such that it's missing removal of nodes which are no longer active (or are faulty)
The genesis file (for sov:staging) I see here https://github.com/sovrin-foundation/sovrin#connecting-to-an-existing-network is 170 lines long, while the latest transaction on pool subledger is number 173 https://indyscan.io/tx/SOVRIN_STAGINGNET/pool/173
I have previously verified this behaviour with my indy network healtchecking tool over here https://github.com/Patrik-Stas/indyscan/pull/219
You can tweak rustiscan/indy-genesis-rs/genesis/sovrin_testnet.ndjson and then run cargo run in rustiscan/indy-health-rs, just tweak it's main.rs to look like this
#[tokio::main]
async fn main() {
env_logger::init();
let (genesis_path, nodes) = ledger_sovrin_testnet();
fetch_a_schema(genesis_path).await;
}Try to run it few times with genesis file provided in my repo, then try to delete last 2 lines of sovrin_testnet.ndjson, try to run couple times and you will see radical difference.
jleach
andrewwhitehead
swcurran
TimoGlastra
WadeBarnes
wadeking98
TL;DR
Although it seems that ledger network performance doesn't directly impact a wallet's ability to complete a transaction (such as accepting a credential), the number of network connections does matter. Wallets with fewer connections to the ledger network tend to complete transactions noticeably faster. This highlights the importance of optimizing the number of network connections for efficiency.
Problem
When using Aries Bifold, BC Wallet, or AFJ to "Accept" a credential, the process can become frustratingly slow on ledgers with many nodes.
Analysis
The table below provides information about test results. These tests were conducted on the
sovrin:stagingenvironment using the LSBC Test credential.In testing process, the same tests were run three times for each platform. However, in the table, we've displayed only the two best results for each platform, with the exception of Orbi Edge.
For Orbi Edge, the first test result is shown, even though it was slower. This initial test showed a notably higher number of network connections. Subsequent tests for Orbi Edge were more optimized.
In the table, you'll find two key columns:
During testing, both Trinsic and Lissi encountered issues when attempting to accept the credential. After waiting for ~60 seconds, an error message appeared.
It was noted via logging that when accepting a credential, AFJ makes a series of network calls, which are logged as follows:
To accurately assess network performance and duration, these calls were replicated using the IndyVDR Proxy and cURL. The results of these tests are documented in the table below.
It is not know what framworks Orbi Edge, Lissi, or Trinsic use.
In the case of Lissi and Trinsic, our observations indicate that they scan all configured ledgers while in the process of accepting a credential. This thorough scanning approach likely played a role in reaching a timeout at approximately 60 seconds, resulting in a test failure.
NOTE Preliminary testing with Trinsic showd it getting similar results to Orbi Edge, however, after a re-install this was not the case and the above results were collected.
Conclusion
This slowness seems to stem from AFJ or IndyVDR making numerous network calls. While each call is quick on its own, the cumulative effect can lead to significant delays when considering response processing.
Our test results reveal some key insights:
Ledger Network Performance: It's worth noting that ledger network performance doesn't seem to significantly impact the results, as evidenced by the efficient performance of IndyVRD Linux and Orbi Edge. Orbi Edge impressively completes transactions in just 3 seconds. Even when we eliminate duplicate network calls from IndyVDR Linux, it achieves similar results.
Number of Network Connections: On the other hand, the number of network connections appears to be a notable factor in test results. Tests that establish fewer connections, possibly the minimum required, tend to complete significantly faster compared to those that create multiple network connections.
In light of these findings, we recommend that AFJ and IndyVDR consider optimizing their ledger network interactions by:
Removing Duplicate Network Calls: Identify and eliminate any duplicate network calls that are part of the same transaction to reduce redundancy.
Batching Queries: Implement a strategy to batch queries, sending them to the same two nodes over the same connection, rather than establishing multiple network connections for each query.
Network Reconciliation: Consider periodic network reconciliation or scheduling intervals for this process, separating it from critical transactions such as accepting a credential or processing proof requests.
Caching Transactions: Explore the possibility of caching ledger transactions, given their immutability, to improve efficiency.
These optimizations could enhance the overall performance of ledger network interactions. Thank you for your attention to these recommendations, which aim to streamline the process for a smoother user experience.
How To Reproduce
Use a demo on Sovrin Test/Staging. If you don't have one, use this email verification service. Watch your firewall logs, time the result. If you have a
pfSensebased router withpfTopyou can run this filter:tcp dst port 9700||9702||9744||9777||9778||9799 and out.