waku-org / js-waku

JavaScript implementation of Waku v2
https://js.waku.org
Apache License 2.0
162 stars 41 forks source link

feat!: use ShardingParams on subscriptions, make Decoder/Encoder auto sharding friendly by default #1958

Closed weboko closed 2 months ago

weboko commented 2 months ago

Problem

Seems latest fleet update is not allowing to subscribe to DefaultPubsubTopic anymore and we should be using shards.

Solution

Use pubsub topic for subscription creation from those determined from shards / contentTopics

Though this PR is huge it includes only following changes:

Notes

Addresses - https://github.com/waku-org/js-waku/issues/1961 Next PR - https://github.com/waku-org/js-waku/pull/1959

github-actions[bot] commented 2 months ago

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 180.93 KB (+0.11% 🔺) 3.7 s (+0.11% 🔺) 13.8 s (-43.29% 🔽) 17.5 s
Waku Simple Light Node 180.85 KB (-0.01% 🔽) 3.7 s (-0.01% 🔽) 22.2 s (+30.04% 🔺) 25.8 s
ECIES encryption 23.08 KB (-0.14% 🔽) 462 ms (-0.14% 🔽) 6.5 s (+11.66% 🔺) 6.9 s
Symmetric encryption 22.55 KB (-0.15% 🔽) 452 ms (-0.15% 🔽) 5.1 s (-20.88% 🔽) 5.6 s
DNS discovery 72.42 KB (0%) 1.5 s (0%) 12.3 s (+28.64% 🔺) 13.7 s
Peer Exchange discovery 74.1 KB (+0.18% 🔺) 1.5 s (+0.18% 🔺) 15.6 s (+64.29% 🔺) 17.1 s
Local Peer Cache Discovery 67.64 KB (0%) 1.4 s (0%) 16.5 s (+14.35% 🔺) 17.9 s
Privacy preserving protocols 38.87 KB (0%) 778 ms (0%) 11.2 s (+2.99% 🔺) 12 s
Waku Filter 111.42 KB (+0.16% 🔺) 2.3 s (+0.16% 🔺) 18.2 s (+93.72% 🔺) 20.5 s
Waku LightPush 110.11 KB (-0.12% 🔽) 2.3 s (-0.12% 🔽) 12.1 s (+9.49% 🔺) 14.3 s
History retrieval protocols 110.72 KB (+0.01% 🔺) 2.3 s (+0.01% 🔺) 15 s (-29.78% 🔽) 17.2 s
Deterministic Message Hashing 4.83 KB (0%) 97 ms (0%) 701 ms (+64.37% 🔺) 798 ms
danisharora099 commented 2 months ago

I might be missing something but I don't fully understand what the PR is solving for. Can you please elaborate, or point to related issues/references?

weboko commented 2 months ago

I might be missing something but I don't fully understand what the PR is solving for. Can you please elaborate, or point to related issues/references?

This problem is addressed - https://github.com/waku-org/js-waku/issues/1961 Happens due to https://discord.com/channels/1110799176264056863/1111539372618166294/1231967635722342540