dmonad / crdt-benchmarks

A collection of CRDT benchmarks
Other
443 stars 29 forks source link

working on bumpning ywasm to latest release #26

Closed dmonad closed 7 months ago

dmonad commented 7 months ago

Hey @Horusiath,

I'm trying to bump Ywasm to the latest release.

I'm running into an issue when calling applyUpdateV2:

RuntimeError: unreachable
    at __rust_start_panic (wasm://wasm/0037b566:wasm-function[1668]:0xb9013)
    at rust_panic (wasm://wasm/0037b566:wasm-function[1566]:0xb8b6d)
    at std::panicking::rust_panic_with_hook::h62090d3fc0630473 (wasm://wasm/0037b566:wasm-function[931]:0xa83c2)
    at std::panicking::begin_panic_handler::{{closure}}::h823c80ef219a166d (wasm://wasm/0037b566:wasm-function[1057]:0xae5a6)
    at std::sys_common::backtrace::__rust_end_short_backtrace::hcc7222b556723586 (wasm://wasm/0037b566:wasm-function[1649]:0xb8f99)
    at rust_begin_unwind (wasm://wasm/0037b566:wasm-function[1348]:0xb67e3)
    at core::panicking::panic_fmt::hd79411a297d06dc8 (wasm://wasm/0037b566:wasm-function[1409]:0xb76c1)
    at core::result::unwrap_failed::h59d2e017e8867fe4 (wasm://wasm/0037b566:wasm-function[1137]:0xb1575)
    at ywasm::apply_update_v2::hdac8c1b389a9d582 (wasm://wasm/0037b566:wasm-function[647]:0x94727)
    at applyUpdateV2 (wasm://wasm/0037b566:wasm-function[1193]:0xb3051)

I logged the input parameters for applyUpdate and the results look okay, nothing weird:

YDoc { __wbg_ptr: 26076832 } Uint8Array(35657) [
    0,   0,   5, 166, 145, 204, 171,  14,   0,   0,  1,   8,
    7,   5,  97, 114, 114,  97, 121,   5,   1,   1,  0,   2,
  176,  93,   1,   1,   0, 125, 143, 194, 184, 154,  9, 125,
  143, 198, 165, 148,  14, 125, 182, 142, 144, 247, 11, 125,
  180, 166, 201, 188,  11, 125, 189, 135, 226, 170,  6, 125,
  141, 232, 180, 250,  11, 125, 159, 151, 142, 192, 13, 125,
  129, 214, 209, 217,   4, 125, 131, 158, 164, 144,  3, 125,
  190, 150, 219, 177,   4, 125, 164, 137, 229, 142, 14, 125,
  130, 201, 207, 221,
  ... 35557 more items
]

Could you please help me debug the issue?

Just run node run.js in benchmarks/ywasm/.

Horusiath commented 7 months ago

Issue is in factory.applyUpdate: https://github.com/dmonad/crdt-benchmarks/blob/8203fde042bddb486f0387957cafb1dd94eb3dac/benchmarks/ywasm/factory.js#L62-L64

ywasm Y.applyUpdateV2(this.ydoc, update) creates a new write transaction. However this method is always executed within the doc.transact context which already had created a write transaction before. Since two write transaction cannot exist at the same time an exception happens (however I agree that there should be more human-friendly error message).

I'll send fixes soon.

Horusiath commented 7 months ago

I'm also adding my results - last column is raw yrs (using cargo criterion), which is only for loose reference and to observe JS<->WASM barrier overhead, as tool and its methodology are different:

yjs ywasm yrs
yjs [B1.1] Append N characters (time) 189 ms 110 ms 13 ms
yjs [B1.2] Insert string of length N (time) 0 ms 0 ms 0ms
yjs [B1.3] Prepend N characters (time) 125 ms 28 ms 5ms
yjs [B1.4] Insert N characters at random positions (time) 131 ms 110 ms 70ms
yjs [B1.5] Insert N words at random positions (time) 136 ms 294 ms 22ms
yjs [B1.6] Insert string, then delete it (time) 1 ms 1 ms
yjs [B1.7] Insert/Delete strings at random positions (time) 156 ms 108 ms 365ms
yjs [B1.8] Append N numbers (time) 159 ms 34 ms 5ms
yjs [B1.9] Insert Array of N numbers (time) 0 ms 1 ms 0ms
yjs [B1.10] Prepend N numbers (time) 114 ms 30 ms 13ms
yjs [B1.11] Insert N numbers at random positions (time) 124 ms 144 ms 70ms
yjs [B2.1] Concurrently insert string of length N at index 0 (time) 1 ms 0 ms 0ms
yjs [B2.2] Concurrently insert N characters at random positions (time) 33 ms 327 ms 205ms
yjs [B2.3] Concurrently insert N words at random positions (time) 57 ms 785 ms 400ms
yjs [B2.4] Concurrently insert & delete (time) 129 ms 2807 ms 630ms
yjs [B3.1] 20√N clients concurrently set number in Map (time) 75 ms 206 ms 27ms
yjs [B3.2] 20√N clients concurrently set Object in Map (time) 79 ms 197 ms 30ms
yjs [B3.3] 20√N clients concurrently set String in Map (time) 224 ms 206 ms 28ms
yjs [B3.4] 20√N clients concurrently insert text in Array (time) 68 ms 226 ms 27ms
yjs [B4] Apply real-world editing dataset (time) 5263 ms 23980 ms 5793ms

We still can improve in many ways.