[BUG]: panic: putting the same entry twice not supported

[bgornicki]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current behavior

P2P microservice spontaneously fails with the following error:

panic: putting the same entry twice not supported

goroutine 165 [running]:
github.com/libp2p/go-libp2p-pubsub/timecache.FirstSeenCache.Add({0xc000e2a960?, 0xc000e2a990?, 0xab8c85?, 0xc000042e88?}, {0xc0027eb890, 0x2b})
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/timecache/first_seen_cache.go:32 +0x28d
github.com/libp2p/go-libp2p-pubsub.(*PubSub).markSeen(0xc000e0cb40, {0xc0027eb890, 0x2b})
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/pubsub.go:1002 +0x102
github.com/libp2p/go-libp2p-pubsub.(*validation).validate(0xc000cffa40, {0xc000290d20, 0x1, 0x0?}, {0xc002954f90, 0x22}, 0xc002c2af80, 0x0)
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:288 +0x11c
github.com/libp2p/go-libp2p-pubsub.(*validation).validateWorker(0xc000cffa40)
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:266 +0x4e
created by github.com/libp2p/go-libp2p-pubsub.(*validation).Start
    /go/pkg/mod/github.com/libp2p/go-libp2p-pubsub@v0.8.3/validation.go:133 +0x5a

It happens randomly every few days, sometimes every few hours. Restarting the node seems to fix the issue until the next occurrence.

Expected behavior

P2P microservice should not fail randomly.

Steps to reproduce

No response

Environment

- OS: Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-60-generic x86_64)
- HW: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz, 32GB RAM

Anything else?

koinos_p2p.log

[mvandeberg]

Interesting... The stack trace is entirely in libp2p.

[bgornicki]

I quickly check stackoverflow for same error and it seems like a bug in libp2p. Maybe bumping to newer version will fix it?

[mvandeberg]

We tried bumping libp2p earlier today to v0.30.0 and it resulted in peer IDs becoming non-deterministic with configured seeds (filed an issue on libp2p). Perhaps a version between v0.24.0 and v0.30.0 would work.

[sgerbino]

This should be resolved in the next release from this PR: #285

If the issue persists feel free to re-open this ticket.