Windows: writing to local file system from browser is slow

[abchatra]

Repro steps

1. Go to vscode.dev in-private window and install makecode extension (https://marketplace.visualstudio.com/items?itemName=ms-edu.pxt-vscode-web)
2. Open Folder and select a local system drive which is empty
3. Click on View Files from the browser dialog to give permission.
4. Run command to "Create an empty project" -> Pick a templete->give a name
5. It takes over one minute to download all dependencies and create files.

Desktop client takes less than 5 seconds

Gif below with side by side comparison with desktop client:

I am unsure if this is browser API issue or VSCode.dev issue. Version: 1.76.2 Commit: ee2b180d582a7f601fa6ecfdad8d9fd269ab1884 User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44 Embedder: vscode.dev

[abchatra]

Original issue was filed here and asked to move to this repo: https://github.com/microsoft/vscode-dev/issues/852

The size of the payload is in 10-15 MB. However, it is not the network speed which is blocking this (Gbps).

[abchatra]

Also, note if we invoke similar project through the route, the load is fast (5-10 seconds)

1. Go to in-private in browser
2. Copy & paste: https://vscode.dev/makecode/_hw6ax2LThPVc
3. Loads within 5-6 seconds.

We implement virtual file system in this case and hence this scenario is not impacted. So likely culprit is the file system interaction.

[bpasero]

Unfortunately the source code for that extension seems closed source and not open source, at least I cannot see it. I wonder if the extension could speed things up by using the file service in parallel and not in sequence.

[bpasero]

I am not able to reproduce slow writes locally. I can write 100mb of data in 5 seconds to disk.

Here is my sample extension code:

import * as vscode from 'vscode';

const alphabet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ\r\n';

function randomChar() {
    return alphabet[randomInt(alphabet.length)];
}

function randomInt(bound: number) {
    return Math.floor(Math.random() * bound);
}

function randomStr(len: number) {
    if (len === null) {
        len = 10;
    }
    return (function () {
        let j, ref;
        const results = [];
        for (
            j = 1, ref = len;
            1 <= ref ? j < ref : j > ref;
            1 <= ref ? j++ : j--
        ) {
            results.push(randomChar());
        }
        return results;
    })().join('');
}

export function activate(context: vscode.ExtensionContext) {

    console.log('Congratulations, your extension "create-random-files" is now active in the web extension host!');

    let disposable = vscode.commands.registerCommand('create-random-files.helloWorld', async () => {
        vscode.window.showInformationMessage('Hello World from Create Random Files in a web extension host!');

        const start = Date.now();
        const promises = [];

        for (let i = 0; i < 1000; i++) {
            const randomFile = vscode.Uri.joinPath(vscode.workspace.workspaceFolders![0].uri, `somefile-${i}.txt`);
            promises.push(vscode.workspace.fs.writeFile(randomFile, new TextEncoder().encode(randomStr(100000))));
        }

        await Promise.allSettled(promises);

        console.log(`Took ${Date.now() - start}ms to create 1000 files`);
    });

    context.subscriptions.push(disposable);
}

export function deactivate() { }

Which I run in a chrome instance via npx @vscode/test-web --extensionDevelopmentPath=$extensionLocation where $extensionLocation is a local web extension I created via yo code.

I do see however 3x slowness when I do not create all the files in parallel but in sequence, so that seems to be something maybe the Make-Code extension could check.

I really need a repro here that is minimal so please come back with steps how to reproduce.

[abchatra]

@bpasero I have given access to this repo.

Apologies, we are in the process of making vscode-makecode opensource. It should be done soon.

@riknoll @jwunderl can you please point to the relevant code here.

[bpasero]

Thanks, waiting for a minimal repro. Note that we cannot really review other peoples code.

[abchatra]

I understand, we will work on small repro. Thank you for the directions .

[jwunderl]

@bpasero When I run the sample extension code above with a new yo code web extension served via npx @vscode/test-web I'm seeing it take 2-3 minutes on my work device when saving to a local folder.

where the folder is empty to start with

I ran it again with the chromium dev tools performance profiler on and it took 252.428 seconds, let me know and I can send over the profile / any other info that can help. The majority of time is listed as 'idle' while the promises were running and populating the folder:

I have a sample I had written up that more closely matches our use case but the result is the same as the sample you gave (just adding in folders and a mkdirp), just leaving collapsed below in case it's helpful.

another sample repro

trimmed down from https://github.com/microsoft/pxt-mkc/blob/master/packages/makecode-core/src/files.ts#L114 with the host handlers for browser defined in https://github.com/microsoft/vscode-makecode/blob/main/src/web/host.ts), as we share the logic for cli & browser implementations. Also, I had tried making the saves run in parallel before as you mentioned but that ended up locking up the browser window completely until the saves were done without noticeably decreasing the total set up time. ```typescript async function bulkSaveTestAsync() { const ws = vscode.workspace.workspaceFolders![0]; const alphabet = "abcdefghijklmnopqrstuvwxyz"; const lorem = new TextEncoder().encode(`Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Pellentesque nec nam aliquam sem et tortor consequat. Sagittis id consectetur purus ut faucibus pulvinar elementum integer. Tristique nulla aliquet enim tortor at. Ornare aenean euismod elementum nisi quis eleifend quam adipiscing vitae. Eget nunc lobortis mattis aliquam faucibus purus. Diam in arcu cursus euismod quis. Pulvinar elementum integer enim neque volutpat ac tincidunt vitae semper. Risus quis varius quam quisque id diam vel quam. Vitae turpis massa sed elementum tempus egestas. Aenean euismod elementum nisi quis eleifend quam. Tellus rutrum tellus pellentesque eu tincidunt tortor aliquam nulla facilisi. Faucibus interdum posuere lorem ipsum dolor sit amet consectetur adipiscing. Ullamcorper malesuada proin libero nunc consequat interdum varius sit. Ultricies mi eget mauris pharetra et ultrices neque ornare. Faucibus a pellentesque sit amet porttitor eget dolor morbi non. Pellentesque elit eget gravida cum. Nisl vel pretium lectus quam id leo in vitae. Feugiat sed lectus vestibulum mattis. Nisi est sit amet facilisis magna etiam tempor orci. Ac feugiat sed lectus vestibulum mattis ullamcorper. Ultrices in iaculis nunc sed augue lacus viverra vitae. Non enim praesent elementum facilisis leo. Amet justo donec enim diam vulputate ut pharetra. Morbi blandit cursus risus at ultrices mi tempus. Enim ut tellus elementum sagittis vitae et leo duis ut. Sed cras ornare arcu dui vivamus arcu. Amet porttitor eget dolor morbi non arcu risus. Blandit cursus risus at ultrices mi tempus imperdiet. Vitae congue mauris rhoncus aenean vel elit. Scelerisque in dictum non consectetur. Ullamcorper morbi tincidunt ornare massa eget egestas purus viverra accumsan. Duis ut diam quam nulla porttitor massa. Aliquam purus sit amet luctus venenatis lectus magna. Fusce ut placerat orci nulla pellentesque dignissim enim sit. Auctor urna nunc id cursus. Sit amet volutpat consequat mauris nunc congue nisi vitae. Justo donec enim diam vulputate ut. `); const resolvePath = (path: string) => vscode.Uri.joinPath(ws.uri, "temp", path); const existsAsync = async (path: string) => { const resolvedPath = resolvePath(path); try { await vscode.workspace.fs.stat(resolvedPath); return true; } catch { return false; } } const start = Date.now(); for (const folder of alphabet) { for (const file of alphabet) { const fPath = `${folder}/${file}.txt`; if (!(await existsAsync(folder))) { // matching a mkdirp as we're saving from a json blob of `{ [fileName]: content }` await vscode.workspace.fs.createDirectory(resolvePath(folder)); } await vscode.workspace.fs.writeFile( resolvePath(fPath), lorem ); } } const duration = Date.now() - start; console.log(`finished writing files after ${duration}ms`) } ```

[bpasero]

Can you also try on a OS such as macOS or Linux, maybe it is Windows related. If it is OS specific then it sounds like a Chrome issue or maybe AntiVirus/Defender and then VS Code will not be able to do anything about it.

[bpasero]

Btw I can confirm it is also much slower for me on Windows, with times around 150s. I can only speculate that there is an issue with the Chrome implementation on Windows that makes this slower, because we have no windows specific code for creating the files.

Here is our code:

https://github.com/microsoft/vscode/blob/1606487f6ebfe224cdbb1d6803241c7326115a43/src/vs/platform/files/browser/htmlFileSystemProvider.ts#L223-L225

One thing we could experiment with is to implement stream based writing, because today we write everything at once (the full contents). However, typically writing everything at once is faster, not slower, just consuming more memory...

[bpasero]

As a starter, I filed a Chrome issue to hear back if we do something wrong: https://bugs.chromium.org/p/chromium/issues/detail?id=1427148

[abchatra]

Thanks so much @bpasero !

[bpasero]

Closing as upstream. See discussion in https://bugs.chromium.org/p/chromium/issues/detail?id=1427148