`Async` file systems are not fully copied

[james-pre]

This issue was first mentioned on Discord:

@Davilarek:

Is there a way I can wait for the FS to sync everything from indexedDB to InMemory? async calls work fine but calls that are sync require some time to have any content sometimes sync calls return empty buffers even after some time... (but async still works fine)

Me:

If you wait for the IndexedDB.ready() That should wait for the sync cache to populate

@Davilarek:

isn't awaiting configure already doing this? https://github.com/zen-fs/core/blob/c33ad77a/src/config.ts#L61

Me:

Yes, perhaps crossCopy is not being awaited in ready then

@Davilarek:

I'm not sure

In my investigation, I found that:

• Async.crossCopy correctly awaits itself
• AsyncStoreFS.ready correctly awaits super.ready
• Async.ready correctly awaits the sync FS
• Async.ready does not await super.ready!

Async.ready:

public async ready(): Promise<this> {
    await this._sync.ready(); // correctly waits for the sync FS to be ready
    // super.ready is missing!
    if (this._isInitialized) {
        return this; // prevents duplicate copying
    }

    try {
        await this.crossCopy('/');
        this._isInitialized = true;
    } catch (e) {
        this._isInitialized = false;
        throw e;
        }
    return this;
}

However, since FileSystem can be passed to Async care must be taken to ensure that super.ready exists.

[james-pre]

This should be resolved in v0.9.4

[Davilarek]

does v0.9.4 fix need any additional changes?

window.ZenFS.configure({
            backend: window.ZenFS_DOM.IndexedDB,
            storeName: "VirtualFS",
            idbFactory: indexedDB,
    }).then(() => {
        if (!window.ZenFS.fs.existsSync("/test.txt")) {
            console.log("creating because missing");
            window.ZenFS.fs.writeFileSync("/test.txt", "hello world!");
            window.ZenFS.fs.mkdirSync("/test123");
            window.ZenFS.fs.mkdirSync("/test123/test456");
            window.ZenFS.fs.writeFileSync("/test123/test456/test.txt", "hello world!");
        }
        else {
            console.log("found");
        }
        console.log(window.ZenFS.fs.readFileSync("/test123/test456/test.txt")); // on first: works, on second: empty
        window.ZenFS.fs.readFile("/test123/test456/test.txt", "utf8", console.log); // always works!
    });

this example here still looks to have the issue

[james-pre]

I have some different results.

On the first run:

• sync read is successful
• async read throws an error: Error: ENOENT: No such file or directory, '/test123/test456/test.txt' On the second run:
• sync read is empty
• async read is successful

After some investigation, I've found that when opening a file from the sync file system, the resulting file's buffer is empty.

This is definitely confusing behavior. I'm looking into it and will hopefully correct the issue soon.

[james-pre]

The reason the error is occurring on the first run is due to the queued operations not being completed when readFile is called. This should be fixed in 9aca101.

[james-pre]

The second issue is caused by crossCopy calling File.writeSync without passing in length, so it defaulted to File.stats.size, which was 0 because it was a newly created file. This should be fixed by 560b307.

[james-pre]

@Davilarek Fixes have been released in v0.9.5. Please let me know if this fixes the issues on your side.

[Davilarek]

Thanks, it looks like this issue is solved.