Emulating files with no writing to disk

AlexCannonball commented 1 month ago

Node.js Version

v18.19.1

NPM Version

10.8.0

Operating System

Windows 10 Pro

Subsystem

child_process, fs, Other

Description

Hello,

I make a VS Code extension (Electron/Node.js application) that uses a Golang linter application to lint files. The linter is a third party executable that accepts only paths to files. It also has a configuration YAML where a path to directory or file is used to enable/disable linting rules for particular files:

  directories:
    exclude:
      - 'some/directory'
  files:
    exclude:
      - 'another/directory/some.file'

The VS Code extension should work on Windows, Linux and OS X.

I'd like to make "lint on type" feature in the VS Code extension, conveying unsaved edited text from IDE to the abovementioned linter. I have to create a temporary file with the current IDE text to give it to the linter.

Because of the configuration YAML the relative path of the temporary file should be equal to the original file.

I've successfully made a prototype via fs.writeFile to create the temporary file and cp.spawn to run the linter.

async function createFile(
  document: TextDocument,
  temporaryDirectory: string,
  method: keyof Executable & ('autofix' | 'lint'),
): Promise<TResult<string, ITemporaryFileError>> {
  const nestedDirectories = workspace.asRelativePath(
    path.dirname(document.fileName),
    false,
  );

  let directory = temporaryDirectory;

  const SAFE_FILE_NAME = 'file.proto';
  let fileName;

  try {
    fileName = path.join(
      directory,
      method === 'autofix' ? SAFE_FILE_NAME : path.basename(document.fileName),
    );

    const {
      constants: { UV_FS_O_FILEMAP, O_CREAT, O_WRONLY },
    } = fs;

    // UV_FS_O_FILEMAP doesn't prevent from writing to disk
    await fs.writeFile(fileName, document.getText(), {
      flag: UV_FS_O_FILEMAP | O_CREAT | O_WRONLY,
    });
  } catch (error) {
    // ...
  }

  return {
    result: 'success',
    value: fileName,
  };
}

import * as cp from 'node:child_process';
import { once } from 'node:events';
import { text } from 'node:stream/consumers';
// ...

const runExecutable: TRunExecutable = async function (
  ...arguments_
): Promise<TResult<IExecResult, IExecFileError>> {
  const process = cp.spawn(...arguments_);

  const stderr = text(process.stderr);
  const stdout = text(process.stdout);

  try {
    const [exitCode] = (await once(process, 'exit')) as unknown[];

    if (exitCode === null) {
      return {
        result: 'error',
        error: { code: ExecFileErrorCodes.Terminated },
      };
    }

    switch (exitCode) {
      case ProtolintExitCode.Clear:
        return {
          result: 'success',
          value: { exitCode, stdout: await stdout },
        };

      case ProtolintExitCode.LintFlags:
      case ProtolintExitCode.OtherErrors:
        return {
          result: 'success',
          value: { exitCode, stderr: await stderr },
        };
    }
  } catch (error) {
    // ...
  }
};

However, in this prototype every IDE text edit causes several excessive disk operations:

Create the necessary temp directories.
Create the temp file.
Remove the temporary file and directories after linting is done.

I have looked for a workaround to reduce/exclude writing to disk, but haven't find one.

IPC from net (Windows named pipes and Unix domain socket) doesn't work. The code snippet for Windows is below:

const PIPE_PREFIX = String.raw`\\?\pipe`;

function mirrorDocument(document: TextDocument): {
  cwd: string;
  fileName: string;
  server: Server;
} {
  const relativePath = workspace.asRelativePath(document.uri, false);
  const pipeName = path.join(
    PIPE_PREFIX,
    path.isAbsolute(relativePath)
      ? path.basename(document.fileName)
      : relativePath,
  );
  const server = createServer({ allowHalfOpen: true, noDelay: true });

  server.listen(pipeName);

  server.on('connection', function (socket) {
    socket.end(document.getText());
    socket.destroy();
  });

  return { cwd: PIPE_PREFIX, fileName: pipeName, server };
}

The linter doesn't accept Unix domain socket as a file path. On Ubuntu I also can't make the socket path equal to file path, it doesn't let path separators.
The linter accepts Windows named pipes as a file path, but I can't cp.spawn the linter using the named pipe as cwd, so the linter settings from YAML config are not applied properly. Attempting to use the named pipe as cwd for cp.spawn causes ENOENT error.

fs.writeFile with UV_FS_O_FILEMAP doesn't exclude writing the temporary directories/file to disk.

I haven't find a way to create a temporary file with FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE flags on Windows via Node.js.

I guess a minimum program is finding a workaround for Windows, because on Linux and OS X it's easier to create tmpfs and set it for the extension's temp files.

Any ideas, how to make it? Thank you.

Minimal Reproduction

No response

Output

No response

Before You Submit

[X] I have looked for issues that already exist before submitting this
[X] My issue follows the guidelines in the README file, and follows the 'How to ask a good question' guide at https://stackoverflow.com/help/how-to-ask

RedYetiDev commented 1 month ago

Hi! While I love your detailed issue report, when looking for suggestions, it's important that you provide snippets of code.

Additionally, StackOverflow is also a valuable for code suggestions.

AlexCannonball commented 1 month ago

@RedYetiDev Hi and thank you for the reply! I've updated the issue with TypeScript code snippets.

RedYetiDev commented 1 month ago

Thanks!

Create the necessary temp directories. Create the temp file. Remove the temporary file and directories after linting is done.

I'm not an expert on code-golfing, so I can't help to much from here, but now oncomers will know what to help with. If you want a faster answer, there's plenty of Code-Golfing communities :-)

nodejs / help