Out of memory errors after creating enough large documents in repo

[georgewsu]

Hi, running in to this error when testing an automerge client with a 10MB test document.

Node.js v18.15.0

Steps to reproduce:

Start the sync server

git clone https://github.com/automerge/automerge-repo-sync-server.git
cd automerge-repo-sync-server
npm install
npm start

Then run the client https://github.com/georgewsu/automerge-client-test/tree/error_recursive_use

git clone https://github.com/georgewsu/automerge-client-test.git
cd automerge-client-test
git checkout error_recursive_use
npm install
npm run client

code excerpt: https://github.com/georgewsu/automerge-client-test/blob/error_recursive_use/src/client.ts

const stringArray: string[] = new Array(arrayLength).fill(stringWith100Characters);
const testJson = {
  stringArray: stringArray
};
const testDoc = A.from(testJson);
const handle1 = repo1.create(testDoc);

Result:

Error: recursive use of an object detected which would lead to unsafe aliasing in rust
    at module.exports.__wbindgen_throw (/Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge-wasm/nodejs/automerge_wasm.cjs:1990:11)
    at wasm_bindgen::throw_str::hc0015d8a92fcec96 (wasm://wasm/00664e02:wasm-function[2065]:0x157683)
    at wasm_bindgen::__rt::borrow_fail::hd13a3345a14a3b49 (wasm://wasm/00664e02:wasm-function[2067]:0x15769f)
    at automerge_rollback (wasm://wasm/00664e02:wasm-function[1668]:0x1545ce)
    at Automerge.rollback (/Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge-wasm/nodejs/automerge_wasm.cjs:544:26)
    at _change (file:///Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge/dist/mjs/stable.js:378:22)
    at Module.from (file:///Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge/dist/mjs/stable.js:176:12)
    at Module.from (file:///Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge/dist/mjs/next.js:98:19)
    at new DocHandle (file:///Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge-repo/dist/DocHandle.js:51:21)
    at #getHandle (file:///Users/gsu/repo2/automerge-client-test/node_modules/@automerge/automerge-repo/dist/Repo.js:244:24)

[georgewsu]

Alternative steps to reproduce:

https://github.com/georgewsu/automerge-client-test/tree/file_error

git checkout file_error
gunzip generated.json.gz
mkdir -p dist
cp generated.json dist/data.json
npm install
npm run client

This branch creates a doc from json file contents. The generated json file was created using https://json-generator.com/ with repeat 10000, 10000.

[acurrieclark]

I haven’t had a chance to verify this myself, but it looks as though you are passing an automerge document in to Repo.create().

create should be used without parameters to create an empty doc handle, or with an object to initialise the document handle structure.

Essentially, you might have better luck if you skip the from call and pass testJson directly in to create.

[georgewsu]

I haven’t had a chance to verify this myself, but it looks as though you are passing an automerge document in to Repo.create().

create should be used without parameters to create an empty doc handle, or with an object to initialise the document handle structure.

Essentially, you might have better luck if you skip the from call and pass testJson directly in to create.

Hi, unfortunately that still doesn't prevent the error from happening. It does save a step and reduce the time it takes at least. Thanks

[georgewsu]

To clarify, it takes multiple documents to trigger the error on the server:

cd automerge-client-test
git checkout file_error
du -sh dist/data.json ; echo "should be several MB"
for i in {1..10}; do npm run client; done

[georgewsu]

fyi, the error message "recursive use of an object detected which would lead to unsafe aliasing in rust" is being thrown from the call to rollback and obscuring the actual underlying error

you can test it out by wrapping the call to state.handle.rollback() in a try/catch block

[georgewsu]

Changed issue title from "Error: recursive use of an object detected which would lead to unsafe aliasing in rust" to "Out of memory errors after creating enough large documents in repo"

[georgewsu]

Same issue as https://github.com/automerge/automerge-repo/issues/358