search

koinos / koinos-p2p

The p2p microservice orchestrates the distribution of blocks and transactions between peers.
MIT License
6 stars 4 forks source link

[BUG]: panic: putting the same entry twice not supported #283

Closed bgornicki closed 10 months ago

bgornicki commented 10 months ago

Is there an existing issue for this?

Current behavior

P2P microservice spontaneously fails with the following error:

panic: putting the same entry twice not supported

goroutine 165 [running]:
github.com/libp2p/go-libp2p-pubsub/timecache.FirstSeenCache.Add({0xc000e2a960?, 0xc000e2a990?, 0xab8c85?, 0xc000042e88?}, {0xc0027eb890, 0x2b})
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/timecache/first_seen_cache.go:32 +0x28d
github.com/libp2p/go-libp2p-pubsub.(*PubSub).markSeen(0xc000e0cb40, {0xc0027eb890, 0x2b})
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/pubsub.go:1002 +0x102
github.com/libp2p/go-libp2p-pubsub.(*validation).validate(0xc000cffa40, {0xc000290d20, 0x1, 0x0?}, {0xc002954f90, 0x22}, 0xc002c2af80, 0x0)
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:288 +0x11c
github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000cffa40)
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:266 +0x4e
created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:133 +0x5a

It happens randomly every few days, sometimes every few hours. Restarting the node seems to fix the issue until the next occurrence.

Expected behavior

P2P microservice should not fail randomly.

Steps to reproduce

No response

Environment

- OS: Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-60-generic x86_64)
- HW: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, 32GB RAM

Anything else?

koinos_p2p.log

mvandeberg commented 10 months ago

Interesting... The stack trace is entirely in libp2p.

bgornicki commented 10 months ago

I quickly check stackoverflow for same error and it seems like a bug in libp2p. Maybe bumping to newer version will fix it?

mvandeberg commented 10 months ago

We tried bumping libp2p earlier today to v0.30.0 and it resulted in peer IDs becoming non-deterministic with configured seeds (filed an issue on libp2p). Perhaps a version between v0.24.0 and v0.30.0 would work.

sgerbino commented 10 months ago

This should be resolved in the next release from this PR: #285

If the issue persists feel free to re-open this ticket.