Support for permessage-deflate [✅/awaiting_dependency_merge]

nox commented 7 years ago

~~This is a mandatory part of the spec, and is thus needed to be Web-compatible.~~

agalakhov commented 7 years ago

Thanks, it will be done. The only reason not having it already is, we don't use it ourselves, and the library was written for our production use.

Note for myself: it is RFC 7692.

nox commented 7 years ago

I think I brainfarted when I thought it is mandatory.

agalakhov commented 7 years ago

At least it is required by Autobahn, that's enough for me :)

Currently tungstenite passes all the Autobahn tests with only two of them as "non-strict", and my plan is to achieve 100% strict (I know what exactly fails). For comparison, ws-rs has 11 non-stricts and others are even worse. So the support for RFC 7692 is needed at least to have 100% green lines here. :)

nox commented 7 years ago

It is actually mandatory for a user agent, see step 9 of https://fetch.spec.whatwg.org/#concept-websocket-establish.

dariost commented 7 years ago

As a suggestion I'd say to implement this using something like flate2 and not libz-sys in order to avoid external dependencies on system libraries

Fedcomp commented 6 years ago

Is it still planned? ws-rs support permessage-deflate but does not support async :(

daniel-abramov commented 6 years ago

Yes, we still plan to work on it, but unfortunately we did not have time to work on it yet as it was lower in priorities than other things which we fixed / improved since then.

AlexDaniel commented 3 years ago

Any updates on this? This is the current state of permessage-deflate in rust's websocket libraries:

websocket does not support permessage-deflate (https://github.com/websockets-rs/rust-websocket/issues/123)
The repo for twist seems to be gone (https://github.com/rustyhorde/twist)
tungstenite does not support permessage-deflate (You are here!)
ws is the only one that supports permessage-deflate, but it is abandoned. There is a security issue that is fixed in its fork parity-ws, but it also does not seem to be actively maintained. Unfortunately, compression in ws-rs no longer works with the newest versions of rustc: https://github.com/housleyjk/ws-rs/issues/330
Can't find info about permessage-deflate in websocket-lite but it probably isn't supported
yarws supports permessage-deflate but only for incoming messages (I need it for outgoing messages)

tinaun commented 3 years ago

https://github.com/snapview/tungstenite-rs/pull/144

daniel-abramov commented 3 years ago

Someone needs to update the #144 to the newest state (resolve the conflicts and fix some remarks in code), once it's done and clean, we could merge it. Unfortunately we don't plan to make changes at this particular place in the nearest future, but if someone makes a PR and/or updates the existing #144 which seems to be [almost] ready (?), we would be glad to review and merge it.

Fedcomp commented 3 years ago

We should probably mention @SirCipher to update pull request :)

SirCipher commented 3 years ago

@application-developer-DA, @Fedcomp, sorry for the delay, we've been updating to Tokio 0.3. The work is nearly done and I'll update the PR soon.

rymnc commented 3 years ago

Do you have an update? Thanks!

SirCipher commented 3 years ago

@rymnc, the PR is still awaiting a review at the moment. I've got a local branch that's updated to the most recent version for both this repository and tokio-tungstenite that I'll commit soon too

rymnc commented 3 years ago

Okay, np thanks for the reply

ignatov commented 3 years ago

Hi, do you have any news about the permessage-deflate support?

daniel-abramov commented 3 years ago

The corresponding PR was closed as the author did not have to time to update it / address the remarks, so he decided to close the PR. The new PRs are welcome ;)

kazk commented 3 years ago

I'm currently working on this. So far, I've got it to pass some autobahn (no parameter support, so client passes 12, server passes both 12 and 13) by layering compression/decompression on top of the existing flow, so it doesn't need as much code, and it's clearly additive. I don't think there's any major breaking changes.

$ git diff --cached --stat src/
 src/error.rs                |  14 ++++-
 src/extensions/deflate.rs   | 217 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/extensions/mod.rs       |  89 ++++++++++++++++++++++++++++
 src/handshake/client.rs     |  87 +++++++++++++++++++++++----
 src/handshake/server.rs     |  28 ++++++++-
 src/lib.rs                  |   1 +
 src/protocol/frame/frame.rs |  11 ++++
 src/protocol/mod.rs         | 137 +++++++++++++++++++++++++++++++++++++++----
 8 files changed, 558 insertions(+), 26 deletions(-)

For now, I'm not planning to implement server_max_window_bits/client_max_window_bits support because that requires flate2/zlib feature, and I don't need them, but it should be easy to add them later with a separate feature flag.

Later, we should be able to support other compressions like snappy too.

kazk commented 3 years ago

Opened #235. Any feedback is appreciated.

TheDan64 commented 1 year ago

@kazk Is there any assistance needed to help get this PR across the finish line?

kazk commented 1 year ago

@TheDan64 It's blocked by https://github.com/hyperium/headers/pull/88. More specifically https://github.com/hyperium/headers/pull/88#discussion_r711390478. Maybe providing some feedback on that will be helpful to get that merged.

Once that's merged, I'll open a new PR from a branch using it, which is a rebased version of #235 with header parsing extracted and with better error handling.

TheDan64 commented 1 year ago

Tried out your rebased branch for our use case and had no issues - appears to be working well. Hope to see this upstreamed sooner than later so that we can use it in production & with the other tungstenite sister libraries

daniel-abramov commented 1 year ago

The implementation by @kazk resides in https://github.com/snapview/tungstenite-rs/tree/permessage-deflate and has been temporarily reverted from a master branch, see the discussion in the PR for more details.

nakedible-p commented 1 year ago

FWIW, I have rebased branches of all the relevant functions all the way to axum-tungstenite (alternative to axum::extract::ws) to use the permessage-deflate. Tested to work fine.

My specific use case needs high performance, and configurable window bits, so there is some hardcoding of window bits as I didn't want to implement full window bits handling yet. I can implement full window bits handling... but I'd want the permessage-deflate implementation by @kazk merged first.

https://github.com/hyperium/headers/compare/master...nakedible-p:hyperium-headers:deflate-release https://github.com/snapview/tungstenite-rs/compare/master...nakedible-p:snapview-tungstenite-rs:deflate-release https://github.com/snapview/tokio-tungstenite/compare/master...nakedible-p:snapview-tokio-tungstenite:deflate-release https://github.com/davidpdrsn/axum-tungstenite/compare/main...nakedible-p:davidpdrsn-axum-tungstenite:deflate-release

SvizelPritula commented 1 year ago

Would it maybe be a good idea to implement the parsing of the Sec-Websocket-Extensions header into this crate? The PR agains headers has shown no progress for over ten months. If the PR ever gets merged, it should be pretty simple to switch over to using that version.

kaszperro commented 9 months ago

Could we make it without headers PR? Such a great feature is blocked because of external dependency for such a long time.

snapview / tungstenite-rs

Support for permessage-deflate [✅/awaiting_dependency_merge] #2