Node.js File size is to big

[DanielcBatista]

Hello, i´m trying to get all files from my GOPRO hero 11. and i´m trying to parse all files to node.js to extract all data to JSON files. I´m getting this error.

node:fs:424
      throw new ERR_FS_FILE_TOO_LARGE(size);
      ^

RangeError [ERR_FS_FILE_TOO_LARGE]: File size (4005420209) is greater than 2 GiB
    at new NodeError (node:internal/errors:387:5)
    at tryCreateBuffer (node:fs:424:13)
    at Object.readFileSync (node:fs:469:14)
    at Object.<anonymous> (C:\dev\gopro-telemetry-master\teste.js:6:17)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:22:47 {
  code: 'ERR_FS_FILE_TOO_LARGE'
}

and this is my code

const gpmfExtract = require('gpmf-extract');
const goproTelemetry = require(`gopro-telemetry`);
const fs = require('fs');

const file = fs.readFileSync('E:\\Gravações\\GoPro\\2024-06-08\\HERO5 Black 1\\GOPR9562.mp4');

gpmfExtract(file)
    .then(extracted => {
        goproTelemetry(extracted, {}, telemetry => {
            fs.writeFileSync('.\\output_path.json', JSON.stringify(telemetry));
            console.log('Telemetry saved as JSON');
        });
    })
    .catch(error => console.error(error));

[JuanIrache]

See here: https://github.com/JuanIrache/gpmf-extract/issues/20 Here: https://github.com/JuanIrache/gopro-telemetry/issues/63 And here: https://github.com/JuanIrache/gpmf-extract/issues/39

[DanielcBatista]

Hey worked! thanks for the help!

[Frysuni]

There is still a way without using FFmpeg in NodeJS, it allows you to download a file via a stream, but be careful, it still loads it into RAM! The developer gives you the opportunity to upload the file yourself to mp4box. Below I gave the code, which is not exactly TypeScript friendly, due to the old age of the libraries, but easily allows you to download a file from a stream without using ffmpeg.

function toArrayBuffer(buffer: Buffer) {
  const arrayBuffer = new ArrayBuffer(buffer.length);
  const view = new Uint8Array(arrayBuffer);
  for (let i = 0; i < buffer.length; ++i) {
    view[i] = buffer[i];
  }
  return arrayBuffer;
}

let fileStart = 0;
const extractResult = await GPMFExtract((isoFile) => {
  const fileReadStream = createReadStream(resolve(your_file_path));

  fileReadStream.on('data', (chunk: Buffer) => {
    const arrayBuffer = toArrayBuffer(chunk);
    if (arrayBuffer.byteLength === 0) throw new Error('File not compatible');

    (arrayBuffer as any).fileStart = fileStart;

    isoFile.appendBuffer(arrayBuffer as any);
    fileStart += chunk.length;
  });

  fileReadStream.on('end', () => isoFile.flush());
}, {
  browserMode: false,
  useWorker: false,
});