search

libp2p / go-libp2p-pubsub

The PubSub implementation for go-libp2p
https://github.com/libp2p/specs/tree/master/pubsub
Other
317 stars 182 forks source link

Performance issue #461

Closed eugenechyrski closed 2 years ago

eugenechyrski commented 2 years ago

pubsub.go#527 creates performance issue. This loop uses excessive processor resources. Feels like this loop would really appreciate some time.Sleep(time.Millisecond) inside, dropped cpu usage from 22 to 0.8 % by me

vyzo commented 2 years ago

Thats the pubsub main loop; no, we are not adding a sleep like that: then your throughput is capped at 1k messages per second.

Can you please be more specific about your perf issues? Sounds like you have a lot of pubsub traffic.

eugenechyrski commented 2 years ago

Ok i combined together some parts. Code here is just to reproduce issue , many things are omited. To reproduce it just run the program twice. By me it leads to 30% cpu usage(corei9) by each of the instances

Gist

vyzo commented 2 years ago

That 30% doesnt say much, you need to see exactly which part of the loop is busy.

Maybe it is the combination of discovery, but i dont see anything to explain it.

Also note that this library is used heavily by many projects and we havent had perf problems.

eugenechyrski commented 2 years ago

I will have some time this weekend to go through it

vyzo commented 2 years ago

thanks! If there is a perf bug we'd certainly want to know and fix it.

On Wed, Dec 15, 2021, 11:22 Eugene Chyrski @.***> wrote:

I will have some time this weekend to go through it

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/libp2p/go-libp2p-pubsub/issues/461#issuecomment-994562903, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAI4SV7G6MXNBBEL5AYCUTURBM3ZANCNFSM5KCFYT3A .

eugenechyrski commented 2 years ago

The issue was caused by the code. Two instances of the pubsub.PubSub has been created. Had not debugged further to see what caused the issue inside, my guess is that one of the loops was spinning without taking any events

vyzo commented 2 years ago

sounds reasonable, thanks for investigating.