paritytech / substrate

Substrate: The platform for blockchain innovators
Apache License 2.0
8.39k stars 2.65k forks source link

chore: update libp2p to 0.52.1 #14429

Closed melekes closed 1 year ago

melekes commented 1 year ago

changelog: https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-v0.52.0

polkadot companion: https://github.com/paritytech/polkadot/pull/7472 cumulus companion: https://github.com/paritytech/cumulus/pull/2838

melekes commented 1 year ago

discovery test is broken 😞 not sure why

melekes commented 1 year ago

looks like the node rejects both protocols (legacy & new):

[2023-07-12T06:06:29Z TRACE multistream_select::protocol] Received message: Protocol(Protocol("/dot/kad"))
[2023-07-12T06:06:29Z DEBUG multistream_select::listener_select] Listener: rejecting protocol: /dot/kad

[2023-07-12T06:06:29Z TRACE multistream_select::protocol] Received message: Protocol(Protocol("/0000000000000000000000000000000000000000000000000000000000000001/test-fork-id/kad"))
[2023-07-12T06:06:29Z DEBUG multistream_select::listener_select] Listener: rejecting protocol: /0000000000000000000000000000000000000000000000000000000000000001/test-fork-id/kad

[2023-07-12T06:06:29Z DEBUG libp2p_kad::behaviour] Request to PeerId("12D3KooWRrAhoqQhm83b3nTJtTdvoRU6EHGbVN1o2vNWMvBJZCtX") in query QueryId(51) failed with Upgrade(NegotiationFailed)
melekes commented 1 year ago

@altonen maybe you have some ideas how to fix https://github.com/paritytech/substrate/pull/14429#issuecomment-1631937183 ?

altonen commented 1 year ago

@altonen maybe you have some ideas how to fix #14429 (comment) ?

Are you able to add debug prints to multistream-select and see what protocols the listener is supporting?

melekes commented 1 year ago

@altonen maybe you have some ideas how to fix #14429 (comment) ?

Are you able to add debug prints to multistream-select and see what protocols the listener is supporting?

yamux and noise

mxinden commented 1 year ago

@melekes is the rejecting node running in Kademlia Mode::Client?

See v0.44.0 changelog for details: https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/CHANGELOG.md#0440

Easiest way to validate is to call Kademlia::set_mode which was introduced in libp2p-kad v0.44.2.

    swarm
        .behaviour_mut()
        .kademlia
        .set_mode(Some(kad::Mode::Server));
melekes commented 1 year ago

@melekes is the rejecting node running in Kademlia Mode::Client?

See v0.44.0 changelog for details: https://github.com/libp2p/rust-libp2p/blob/master/protocols/kad/CHANGELOG.md#0440

Easiest way to validate is to call Kademlia::set_mode which was introduced in libp2p-kad v0.44.2.

    swarm
        .behaviour_mut()
        .kademlia
        .set_mode(Some(kad::Mode::Server));

ok, that explains everything. thanks 🙏

michalkucharczyk commented 1 year ago

codewise it looks good, however I am not familiar with all the logic.

melekes commented 1 year ago

bot rebase

paritytech-processbot[bot] commented 1 year ago

Rebased

tomaka commented 1 year ago

While this is not my problem anymore, I find it frustrating that every update of a libp2p version seems to require a +1000 -1000 pull request that understandably takes a long time to write, review, and merge.

I find it a bit ridiculous that a very critical feature is blocked on a simple version bump, but this simple version bump comes bundles with tons of breaking changes.

altonen commented 1 year ago

We all want WebRTC merged asap but this is the reality we find ourselves in and I don't think anybody likes these PRs too much. Do you have a suggestion?

melekes commented 1 year ago
[2023-07-19 08:49:06] [2023-07-19T08:49:06Z INFO  diener::patch] Adding patch for `polkadot`.
[2023-07-19 08:49:08] error: failed to select a version for `sha2`.
[2023-07-19 08:49:08]     ... required by package `sp-core-hashing v9.0.0 (/builds/parity/mirrors/substrate/primitives/core/hashing)`
[2023-07-19 08:49:08]     ... which satisfies path dependency `sp-core-hashing` of package `pallet-alliance v4.0.0-dev (/builds/parity/mirrors/substrate/frame/alliance)`
[2023-07-19 08:49:08]     ... which satisfies git dependency `pallet-alliance` (locked to 4.0.0-dev) of package `collectives-polkadot-runtime v1.0.0 (/builds/parity/mirrors/substrate/companions/cumulus/parachains/runtimes/collectives/collectives-polkadot)`
[2023-07-19 08:49:08]     ... which satisfies path dependency `collectives-polkadot-runtime` (locked to 1.0.0) of package `collectives-polkadot-integration-tests v0.1.0 (/builds/parity/mirrors/substrate/companions/cumulus/parachains/integration-tests/emulated/collectives/collectives-polkadot)`
[2023-07-19 08:49:08] versions that meet the requirements `^0.10.7` are: 0.10.7
[2023-07-19 08:49:08] 
[2023-07-19 08:49:08] all possible versions conflict with previously selected packages.
[2023-07-19 08:49:08] 
[2023-07-19 08:49:08]   previously selected package `sha2 v0.10.6`
[2023-07-19 08:49:08]     ... which satisfies dependency `sha2 = "^0.10"` (locked to 0.10.6) of package `k256 v0.13.1`
[2023-07-19 08:49:08]     ... which satisfies dependency `k256 = "^0.13.0"` (locked to 0.13.1) of package `frame-support v4.0.0-dev (/builds/parity/mirrors/substrate/frame/support)`
[2023-07-19 08:49:08]     ... which satisfies git dependency `frame-support` (locked to 4.0.0-dev) of package `asset-hub-kusama-integration-tests v1.0.0 (/builds/parity/mirrors/substrate/companions/cumulus/parachains/integration-tests/emulated/assets/asset-hub-kusama)`

cumulus companion fails w/ ^

thomaseizinger commented 1 year ago

While this is not my problem anymore, I find it frustrating that every update of a libp2p version seems to require a +1000 -1000 pull request that understandably takes a long time to write, review, and merge.

I find it a bit ridiculous that a very critical feature is blocked on a simple version bump, but this simple version bump comes bundles with tons of breaking changes.

You will appreciate then that lots of engineering hours have already gone and will continue to go into making as few breaking changes as possible:

You can discover more things by following the links in the above issues. Esp. the 0.52 release hardened many many APIs in rust-libp2p that allow us to make improvements without making more breaking changes :)

melekes commented 1 year ago

bot merge

paritytech-processbot[bot] commented 1 year ago

Error: Github API says https://github.com/paritytech/cumulus/pull/2838 is not mergeable

melekes commented 1 year ago

bot merge

paritytech-processbot[bot] commented 1 year ago

Error: "Check reviews" status is not passing for https://github.com/paritytech/cumulus/pull/2838

melekes commented 1 year ago

bot merge