enhancement: Initializing conversations using `newConversation` takes 20-60 seconds

xmtp / xmtp-js

XMTP client SDKs, content types, and other packages written in TypeScript

https://xmtp.org/docs

211 stars 40 forks source link

enhancement: Initializing conversations using `newConversation` takes 20-60 seconds #480

Open fabriguespe opened 11 months ago

fabriguespe commented 11 months ago

Describe

Steps to Reproduce:

Use a wallet with thousands of conversations (around 7,000 to 10,000).
Attempt to initialize a new conversation using newConversation.

Expected Behavior:

Initialization should complete in a more reasonable time frame (milliseconds to a couple of seconds).

Actual Behavior:

It takes 20-60 seconds to initialize a new conversation.

Observations:

Redis caching has improved the time from 5-8 minutes to 20-60 seconds, but it's still not optimal.
This issue seems to affect wallets with a large number of conversations.

rygine commented 10 months ago

@fabriguespe can you provide a way to reproduce this?

fabriguespe commented 10 months ago

I could simulate thousands of wallet messages and measure conversation creation delays progressively.

is there a way to test this scenario with our benchmarking tools?

cc @daria-github

seeARMS commented 6 months ago

I'm reasonably confident we're running into this issue at paragraph.xyz.

We delivered ~100k XMTP messages to >6k unique wallets this quarter. Lately we've been timing out on newConversation. For some reason this also seems to be blocking the NodeJS event loop? It's preventing other Express requests from being handled when newConversation stalls.