Initial set of changes for dilation

[vu3rdd]

These commits add basic types etc for the manager state machine, makes dilation a command line option, refactors some non-dilation code a bit. The client can exchange please message with the python dilation client and is able to receive connection-hints message.

The plan is to send PRs so that we minimize the size of the subsequent PRs and make it ready for review and merge them as the development progresses rather than wait till the end and send a huge PR that would be hard to review.

Keep in mind that we (@ewanas, @vu3rdd, @wuan) are rust newbies, so expect parts of the code to be non-idiomatic.

[codecov[bot]]

Codecov Report

Patch coverage: 60.07% and project coverage change: +1.00 :tada:

Comparison is base (e6ddc75) 44.21% compared to head (008db1f) 45.21%.

:exclamation: Current head 008db1f differs from pull request most recent head ff78cbb. Consider uploading reports for the commit ff78cbb to get more accurate results

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #196 +/- ## ========================================== + Coverage 44.21% 45.21% +1.00% ========================================== Files 18 23 +5 Lines 2576 2771 +195 ========================================== + Hits 1139 1253 +114 - Misses 1437 1518 +81 ``` | [Impacted Files](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole) | Coverage Δ | | |---|---|---| | [cli/src/main.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-Y2xpL3NyYy9tYWluLnJz) | `0.00% <0.00%> (ø)` | | | [src/core/server\_messages.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2NvcmUvc2VydmVyX21lc3NhZ2VzLnJz) | `67.64% <ø> (ø)` | | | [src/dilated\_transfer/mod.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2RpbGF0ZWRfdHJhbnNmZXIvbW9kLnJz) | `0.00% <0.00%> (ø)` | | | [src/forwarding.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2ZvcndhcmRpbmcucnM=) | `0.00% <0.00%> (ø)` | | | [src/transit.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL3RyYW5zaXQucnM=) | `74.75% <ø> (ø)` | | | [src/dilation/manager.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2RpbGF0aW9uL21hbmFnZXIucnM=) | `49.42% <49.42%> (ø)` | | | [src/dilation/events.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2RpbGF0aW9uL2V2ZW50cy5ycw==) | `50.00% <50.00%> (ø)` | | | [src/lib.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2xpYi5ycw==) | `42.85% <50.00%> (+2.85%)` | :arrow_up: | | [src/core.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2NvcmUucnM=) | `74.19% <69.23%> (-3.83%)` | :arrow_down: | | [src/core/protocol.rs](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole#diff-c3JjL2NvcmUvcHJvdG9jb2wucnM=) | `87.87% <87.87%> (ø)` | | | ... and [5 more](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole) | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/magic-wormhole/magic-wormhole.rs/pull/196/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=magic-wormhole)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[wuan]

These are some questions raised by @piegamesde which we talked about and the solutions we agreed on:

The core module knows as little as possible about Dilation, because Dilation generally is opt-in (only thing it needs to know is to tell the other side "no I don't Dilate" if necessary, but even that should be a no-op because of backwards compatibility …)

• Finding: Core module contains VersionsMessage which has properties can_dilate and dilation_abilities.
  • Ignore for now
• Finding: Wormhole object has method dilate()
  • Encapsulate within feature flag

Dilation knows nothing about file transfer, because it may be used as the basis for any application protocol

• Finding We have an enum FileTransferV2Mode in dilation/mod.rs (Defining modes of file transfer application), AppVersions (in dilation/mod.rs) references DilatedTransfer (references FileTransferV2Mode)
  • Create a dilated file transfer module for these. It should be a module outside of dilation so that we only use the public API of dilation. This can be merged with the upcoming file transfer v2 module later.

Applications who use file transfer do not have to interact with Dilation. They just say "I want v2" and stuff just happens for them

• As of now, we switch to dilation if both sides can handle it, but only for the receive action.
• There is already a prototype created by @piegamesde
• For now
  • Dilated file transfer module should assume that dilation is available. Negotiation part should be done elsewhere.
  • In the end there should be a file transfer which uses v2 (with dilation) if possible and fall back to v1.

[meejah]

Dilation generally is opt-in (only thing it needs to know is to tell the other side "no I don't Dilate" if necessary

There's no reason to say "I don't do Dilation": if a peer doesn't indicate support for something, it doesn't support it.

Finding: Wormhole object has method dilate() Encapsulate within feature flag

Applications will still want to opt-in to Dilation at runtime.

That is, plenty of "wormhole-using" applications may have no use or need for Dilation (e.g. magic-folder invites never use a bulk-data connection) so it seems you'd still want both things to me (a feature-flag, but also a dilate() or similar method for the application to signal "I actually want to Dilate this wormhole").

[piegamesde]

There's no reason to say "I don't do Dilation": if a peer doesn't indicate support for something, it doesn't support it.

You're totally right, that was just bad wording.

Applications will still want to opt-in to Dilation at runtime.

This is covered by enabling the dilation feature flag and then (not) calling the dilate() method. Without the dilation feature, that method simply won't exist then.

[piegamesde]

The CI failure may be nightly, but since there has been a new stable release since this was committed it might fail there too.

A quick dig at dependencies' issues tells me that updating the dependencies / cargo update should fix the CI failure.

[wuan]

Force pushed the compacted branch. The original branch is still available: https://github.com/LeastAuthority/magic-wormhole.rs/tree/dilation-1-original