files.write buffering issue

[anywhichway]

• Version: js-ipfs version: 0.17.0-e8b7b66bfa98c2a1c0d0bfc19f698d7d00b6c888 interface-ipfs-core version: ^0.157.0 ipfs-http-client version: undefined Repo version: 12 System version: x64/win32 Node.js version: v16.17.0 Commit: e8b7b66bfa98c2a1c0d0bfc19f698d7d00b6c888

• Platform: Windows 64-bit 10.0.22000

• Subsystem: files

Severity:

High - API breakage

Description:

• What I did

1. Wrote a stringified JavaScript object to a Mutable Filesystem file usings files.write
2. Read the data in the file and converted to string
3. Attempted to parse the string using JSON.parse
4. Wrote a different stringified JavaScript object to the same Mutable Filesystem file usings files.write
5. Read the data in the file and converted to string
6. Attempted to parse the string using JSON.parse

• What happened

Second parse attempt failed due to invalid JSON

• What I expected to happen

Second parse attempt to succeed

• Work around

Remove the file and re-create

• Hypothesis

1. There is some type of shared buffer that is not getting reset properly
2. An EOF marker is being used in files and the EOF is put in the wrong place

Steps to reproduce the error:

import {create} from "ipfs";

const ipfs = await create(); // create an ipfs instance and enhance it to use ipvfs
// use standard ipfs file function rm to remove the file
try {
    await ipfs.files.rm("/write-bug.txt");
} catch(e) {

}

const o1 = [{"path":"QmScjZmC4J4ZHq6bGTUyYSESfTKDhxo8X7o3QShSawTsqi","hash":"f7a67e7a0a50e87e59713999562d06cc3d2511709c0a3ded8020d8247e47251c","version":1,"kind":"String","delta":[],"btime":1673137076140,"mtime":1673137076140},{"hash":"4fe36dd2fd280cbdd9414f3efa61d2b49116453e7edad0316b8b6be1d1c64817","version":2,"kind":"String","delta":[[17,1,""],[13,4,"aul!"]],"mtime":1673137076424},{"hash":"0c8a635762b80e327d384f660387f3acc5f24363de54366404e4a391260fd5c5","version":3,"kind":"String","delta":[[12,1,"m"],[14,2,"ry"]],"mtime":1673137092227}];
const o2 = [{"path":"QmP1MKmdnpz2c1WVZRzx6VkDS2Y7ueLnqhn7J5xMeC3QhD","hash":"4fe36dd2fd280cbdd9414f3efa61d2b49116453e7edad0316b8b6be1d1c64817","rebased":[[1673136578323,2]],"version":2,"kind":"String","delta":[],"mtime":1673136578323,"btime":1673136569845},{"hash":"0c8a635762b80e327d384f660387f3acc5f24363de54366404e4a391260fd5c5","version":2,"kind":"String","delta":[[12,1,"m"],[14,2,"ry"]],"mtime":1673136578323}]
await ipfs.files.write("/write-bug.txt",JSON.stringify(o1),{create:true});
let s1 = "";
for await(const chunk of ipfs.files.read("/write-bug.txt")) {
    s1 += chunk.toString();
}
console.log(JSON.parse(s1));
//JSON.parse(JSON.stringify(o2)); confirm stringify and parse work for object in question
//await ipfs.files.rm("/write-bug.txt"); // work around
await ipfs.files.write("/write-bug.txt",JSON.stringify(o2),{create:true});
let s2 = "";
for await(const chunk of ipfs.files.read("/write-bug.txt")) {
    s2 += chunk.toString();
}
console.log(JSON.parse(s2));

[anywhichway]

This is not an issue. I was unaware of the truncate flag for writing a file. So my hypothesis was correct, I was just not providing the flag.