cBournhonesque / lightyear

A networking library to make multiplayer games for the Bevy game engine
https://cbournhonesque.github.io/lightyear/book
Apache License 2.0
437 stars 45 forks source link

WebTransport connection rejected on FF 124-127 #251

Open simbleau opened 5 months ago

simbleau commented 5 months ago

According to https://caniuse.com/webtransport, FF 124 is supported?

On Firefox 124, MacOS (Silicon), I get a vague WebTransport connection rejected error.

I tried connecting from [::1]:8080, localhost:8080, and 127.0.0.1:8080 with Trunk, no dice.

Chrome works fine.

Stack trace:

INFO lightyear/src/client/networking.rs:385 calling connect on netclient [simple_box-1b3dc2302ae3b879.js:494:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
INFO lightyear/src/transport/webtransport/client_wasm.rs:39 Starting client webtransport task with server url: https://127.0.0.1:5000/ [simple_box-1b3dc2302ae3b879.js:494:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
INFO lightyear/src/connection/netcode/client.rs:521 client connecting to server 127.0.0.1:5000 [1/1] [simple_box-1b3dc2302ae3b879.js:494:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
INFO lightyear/src/transport/webtransport/client_wasm.rs:73 Starting webtransport io thread [simple_box-1b3dc2302ae3b879.js:494:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
panicked at lightyear/src/transport/webtransport/client_wasm.rs:84:18:
failed to connect to server: Custom { kind: Other, error: "failed to connect to server: Error(JsValue(WebTransportError: WebTransport connection rejected\n))" }

Stack:

__wbg_get_imports/imports.wbg.__wbg_new_abda76e883ba8a5f@http://[::1]:8080/simple_box-1b3dc2302ae3b879.js:519:21
console_error_panic_hook::hook::hbd095300cc1546a2@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[14621]:0x1772630
core::ops::function::Fn::call::h29984b27337498f0@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[107601]:0x231d087
std::panicking::rust_panic_with_hook::h1e6ac5d404b8e31b@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[34675]:0x1f180c6
std::panicking::begin_panic_handler::{{closure}}::h24b0f4622f2766a5@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[40537]:0x2047936
std::sys_common::backtrace::__rust_end_short_backtrace::h19f35d272c126e7c@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[106088]:0x2319bff
rust_begin_unwind@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[53591]:0x21c0d8c
core::panicking::panic_fmt::h87755523850ece9e@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[55819]:0x21e4894
core::result::unwrap_failed::hd86007cff22dcd83@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[43586]:0x20bdd66
bevy_tasks::single_threaded_task_pool::TaskPool::spawn::{{closure}}::h17191fcf8a288f1b@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[3063]:0xb7431b
wasm_bindgen_futures::queue::Queue::new::{{closure}}::hb4c8b5dc07879a85@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[13464]:0x16b4015
<dyn core::ops::function::FnMut<(A,)>+Output = R as wasm_bindgen::closure::WasmClosure>::describe::invoke::h0154140b370f79e4@http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm:wasm-function[61984]:0x222cf90
__wbg_adapter_84@http://[::1]:8080/simple_box-1b3dc2302ae3b879.js:265:10
real@http://[::1]:8080/simple_box-1b3dc2302ae3b879.js:210:20

[simple_box-1b3dc2302ae3b879.js:535:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
Uncaught RuntimeError: unreachable executed
    __wbg_adapter_84 http://[::1]:8080/simple_box-1b3dc2302ae3b879.js:265
    real http://[::1]:8080/simple_box-1b3dc2302ae3b879.js:210
[simple_box-1b3dc2302ae3b879_bg.wasm:36840801:1](http://[::1]:8080/simple_box-1b3dc2302ae3b879_bg.wasm)
Uncaught (in promise) 
WebTransportError { source: "session", streamErrorCode: null, name: "WebTransportError", message: "WebTransport connection rejected", code: 0, result: 0, filename: "", lineNumber: 0, columnNumber: 0, data: null }
​
code: 0
​
columnNumber: 0
​
data: null
​
filename: ""
​
lineNumber: 0
​
message: "WebTransport connection rejected"
​
name: "WebTransportError"
​
result: 0
​
source: "session"
​
stack: ""
​
streamErrorCode: null
​
<prototype>: WebTransportErrorPrototype { source: Getter, streamErrorCode: Getter, stack: "", … }

ERROR  No work has been submitted for this frame log.target = "wgpu_core::present";
log.module_path = "wgpu_core::present";
log.file = "/Users/simbleau/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.19.3/src/present.rs";
log.line = 341; 156 [simple_box-1b3dc2302ae3b879.js:494:21](http://[::1]:8080/simple_box-1b3dc2302ae3b879.js)
simbleau commented 5 months ago

Furthermore, simply typing this in the console also fails:

>> new WebTransport('https://localhost:5000')
WebTransport { ready: Promise { "pending" }, reliability: "pending", congestionControl: "default", closed: Promise { "pending" }, datagrams: WebTransportDatagramDuplexStream, incomingBidirectionalStreams: ReadableStream, incomingUnidirectionalStreams: ReadableStream }

Uncaught (in promise) 
WebTransportError { source: "session", streamErrorCode: null, name: "WebTransportError", message: "WebTransport connection rejected", code: 0, result: 0, filename: "", lineNumber: 0, columnNumber: 0, data: null }
​
code: 0
​
columnNumber: 0
​
data: null
​
filename: ""
​
lineNumber: 0
​
message: "WebTransport connection rejected"
​
name: "WebTransportError"
​
result: 0
​
source: "session"
​
stack: ""
​
streamErrorCode: null
​
<prototype>: WebTransportErrorPrototype { source: Getter, streamErrorCode: Getter, stack: "", … }
simbleau commented 5 months ago

More troubleshooting: FF 125 (Stable) also does not work.

simbleau commented 5 months ago

Also doesn't work on FF 127 (Nightly)

Nul-led commented 5 months ago

Looks like a cert issue

Nul-led commented 5 months ago

Im not aware of any issues with FF webtransport using selfsigned certs.

@cBournhonesque im pretty sure we tested FF too?

Though since the api is still unstable there is the possibility of flags being set wrongly as well as recent api changes in FF (unlikely, but possible i guess).

simbleau commented 5 months ago

Looks like a cert issue

What gives you that information from the stack trace? I might not be seeing it.

Also, it works on Chrome, as mentioned. So I'm dually confused then if it's a cert issue.

I used the generate script.

Nul-led commented 5 months ago

Its the most likely explanation since the issue occurs during the handshake.

Definitely requires some more testing to confirm though!

cBournhonesque commented 5 months ago

Thanks for the report; @Nul-led I haven't tested in Firefox, no.

I'm not entirely sure if the self-signed certificate mechanism is intended for production settings, see this: https://github.com/BiagioFesta/wtransport/issues/130

According to this: https://wpt.fyi/results/webtransport?label=experimental&label=master&aligned serverCertificateHashes should be supported in FF 127

simbleau commented 5 months ago

According to this: https://wpt.fyi/results/webtransport?label=experimental&label=master&aligned serverCertificateHashes should be supported in FF 127

I tested FF 127 (Nightly), mentioned earlier. Specifically,

The first connection gives this error: (related to https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream/getReader ?)

INFO lightyear/src/connection/netcode/client.rs:521 client connecting to server 127.0.0.1:5000 [1/1] [simple_box-26e93b1803a3aab7.js:494:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
INFO lightyear/src/transport/webtransport/client_wasm.rs:73 Starting webtransport io thread [simple_box-26e93b1803a3aab7.js:494:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
Uncaught TypeError: ReadableStream.getReader: Trying to read with incompatible controller
    __wbg_getReader_8788325bd5966555 http://localhost:8080/simple_box-26e93b1803a3aab7.js:1878
    __wbg_adapter_84 http://localhost:8080/simple_box-26e93b1803a3aab7.js:265
    real http://localhost:8080/simple_box-26e93b1803a3aab7.js:210
[simple_box-26e93b1803a3aab7.js:1878:37](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
INFO lightyear/src/connection/netcode/client.rs:453 client connect failed. connection request timed out

Trying to connect a second times gives the common error:

O lightyear/src/connection/netcode/client.rs:521 client connecting to server 127.0.0.1:5000 [1/1] [simple_box-26e93b1803a3aab7.js:494:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
INFO lightyear/src/transport/webtransport/client_wasm.rs:73 Starting webtransport io thread [simple_box-26e93b1803a3aab7.js:494:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
panicked at lightyear/src/transport/webtransport/client_wasm.rs:84:18:
failed to connect to server: Custom { kind: Other, error: "failed to connect to server: Error(JsValue(WebTransportError: WebTransport connection rejected\n))" }

Stack:

__wbg_get_imports/imports.wbg.__wbg_new_abda76e883ba8a5f@http://localhost:8080/simple_box-26e93b1803a3aab7.js:519:21
console_error_panic_hook::hook::hbd095300cc1546a2@http://localhost:8080/simple_box-26e93b1803a3aab7_bg.wasm:wasm-function[14621]:0x1772630
core::ops::function::Fn::call::h29984b27337498f0@http://localhost:8080/simple_box-26e93b1803a3aab7_bg.wasm:wasm-function[107601]:0x231d087
std::panicking::rust_panic_with_hook::h1e6ac5d404b8e31b@http://localhost:8080/simple_box-26e93b1803a3aab7_bg.wasm:wasm-function[34675]:0x1f180c6
std::panicking::begin_panic_handler::{{closure}}::h24b0f4622f2766a5@http://localhost:8080/simple_box-26e93b1803a3aab7_bg.wasm:wasm-function[40537]:0x2047936
std::sys_common::backtrace::__rust_end_short_backtrace::h19f35…
[simple_box-26e93b1803a3aab7.js:535:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
Uncaught RuntimeError: unreachable executed
[simple_box-26e93b1803a3aab7_bg.wasm:36840801:1](http://localhost:8080/simple_box-26e93b1803a3aab7_bg.wasm)
Uncaught (in promise) 
WebTransportError { source: "session", streamErrorCode: null, name: "WebTransportError", message: "WebTransport connection rejected", code: 0, result: 0, filename: "", lineNumber: 0, columnNumber: 0, data: null }

ERROR  No work has been submitted for this frame log.target = "wgpu_core::present";
log.module_path = "wgpu_core::present";
log.file = "/Users/simbleau/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.19.3/src/present.rs";
log.line = 341; 179 [simple_box-26e93b1803a3aab7.js:494:21](http://localhost:8080/simple_box-26e93b1803a3aab7.js)
simbleau commented 5 months ago

Its the most likely explanation since the issue occurs during the handshake.

Definitely requires some more testing to confirm though!

How can I help to narrow that down?

Given FF 127 which is supposed to support these certs (thanks @cBournhonesque) doesn't work, does that mean it's a different issue?

cBournhonesque commented 5 months ago

Opened this: https://github.com/BiagioFesta/wtransport/issues/166

MOZGIII commented 5 months ago

Note FF doesn't support serverCertificateHashes according to MDN

image
MOZGIII commented 5 months ago

I would highly recommend using ACME/LetsEncrypt provided certificates, unless you actually need to subvert Web PKI for security reasons.

simbleau commented 5 months ago

Anyone that feels comfortable with testing this with a LetsEncrypt/ACME cert?

MOZGIII commented 5 months ago

I have other wtransport/xwt project that works on FF with ACME and no cert hashes. That said, it still crashes on FF but due to other bug in bevy rendering... So, things are rough :D

cBournhonesque commented 4 months ago

@Nul-led did you test that it works on FF?

Nul-led commented 4 months ago

@cBournhonesque didnt we confirm that server hashes arent supported by FF? whats the point in keeping this open then? maybe close as not planned then 🤔

cBournhonesque commented 4 months ago

I'd like to confirm if possible that examples can work on FF if we use a LetsEncrypt/ACME certificate

ansemjo commented 4 months ago

According to this: https://wpt.fyi/results/webtransport?label=experimental&label=master&aligned serverCertificateHashes should be supported in FF 127

Following this rabbit hole into the linked issued and hg changesets seems to indicate that the parameter should indeed be supported in current Firefox. This is wonderful news.

I came here during an entirely unrelated search and am not a user of this library. Though if I may chime in with my previous WebTransport experiences:

Nul-led commented 4 months ago

@ansemjo thanks for the insights :)

ansemjo commented 4 months ago

I just checked in another project of mine and serverCertificateHashes does indeed work in Firefox 126: image That uses an entirely ephemeral certificate generated on-the-fly in a Go server.

I'm afraid, I can't help you with this issue at hand because I'm not familiar with this library. But it's not because modern Firefox doesn't support this option in general.

Nul-led commented 4 months ago

I just checked in another project of mine and serverCertificateHashes does indeed work in Firefox 126: image That uses an entirely ephemeral certificate generated on-the-fly in a Go server.

I'm afraid, I can't help you with this issue at hand because I'm not familiar with this library. But it's not because modern Firefox doesn't support this option in general.

All good, we'll figure it out. Thx for the heads-up tho, its appreciated.

BiagioFesta commented 2 months ago

https://github.com/BiagioFesta/wtransport/pull/192 on wtransport should fix the firefox issue