googleapis / google-api-nodejs-client

Google's officially supported Node.js client library for accessing Google APIs. Support for authorization and authentication with OAuth 2.0, API Keys and JWT (Service Tokens) is included.
https://googleapis.dev/nodejs/googleapis/latest/
Apache License 2.0
11.42k stars 1.92k forks source link

Failed export googledocs : This file is too large to be exported. #3446

Open SoniSuciadi opened 8 months ago

SoniSuciadi commented 8 months ago

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

1) Is this a client library issue or a product issue? This is the client library for . We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the [ Support page]() to reach the most relevant engineers.

2) Did someone already solve this?

3) Do you have a support contract? Please create an issue in the support console to ensure a timely response.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

import { readFileSync } from 'fs';
import { google } from 'googleapis';
import path from 'path';
import { Readable } from 'stream';

export const bufferToStream = (buffer: Buffer): Readable => {
  const stream = new Readable();
  stream.push(buffer);
  stream.push(null);
  return stream;
};
export const createEmptyStream = (): Readable => {
  const stream = new Readable();
  stream._read = () => {};
  stream.push(null);
  return stream;
};

 export async function downloadPDFDoc(fileId: string): Promise<Buffer> {
  try {
    const KEYFILEPATH = path.join(__dirname, 'happyhomes-360410-4644e17e212c.json');
    const credentials = JSON.parse(readFileSync(KEYFILEPATH, 'utf8'));

    const auth = new google.auth.GoogleAuth({
      credentials,
      scopes: ['https://www.googleapis.com/auth/drive'],
    });
    const drive = google.drive({
      version: 'v3',
      auth,
    });
    const response = await drive.files.export(
      {
        fileId: fileId,
        mimeType: 'application/pdf',
      },
      { responseType: 'stream' },
    );

    return new Promise((resolve, reject) => {
      const chunks: Buffer[] = [];
      response.data.on('data', (chunk: Buffer) => {
        chunks.push(chunk);
      });
      response.data.on('end', () => {
        console.log('File download completed.');
        resolve(Buffer.concat(chunks));
      });
      response.data.on('error', (err: Error) => {
        console.error('Error downloading file:', err);
        reject(err);
      });
    });
  } catch (error) {
    console.error('The API returned an error: ' + error);
    throw error;
  }
}```

The total documents I exported were 18 MB, and when I ran this service, I got this error
```js GaxiosError: {
  "error": {
    "code": 400,
    "message": "This file is too large to be exported.",
    "errors": [
      {
        "message": "This file is too large to be exported.",
        "domain": "global",
        "reason": "badRequest",
        "location": "export.size",
        "locationType": "other"
      }
    ]
  }
}

    at Gaxios._request (/Users/sonisuciadi/Happyhomes/rem-be/node_modules/gaxios/src/gaxios.ts:174:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at JWT.requestAsync (/Users/sonisuciadi/Happyhomes/rem-be/node_modules/google-auth-library/build/src/auth/oauth2client.js:405:18)
    at downloadPDFDoc (/Users/sonisuciadi/Happyhomes/rem-be/src/helpers/google-docs/downloadPDFDoc.ts:57:26)
    at publishOfferDocController (/Users/sonisuciadi/Happyhomes/rem-be/src/controllers/project/procurement/procurementDetailOutputJob.controllers.ts:241:32) {
  config: {
    url: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf',
    method: 'GET',
    responseType: 'stream',
    userAgentDirectives: [ [Object] ],
    paramsSerializer: [Function (anonymous)],
    headers: {
      'x-goog-api-client': 'gdcl/7.0.1 gl-node/20.6.1',
      'Accept-Encoding': 'gzip',
      'User-Agent': 'google-api-nodejs-client/7.0.1 (gzip)',
      Authorization: 'Bearer ya29.c.c0AY_VpZi_y24RI-304tmmNmbUcghDfre0a7qnssZ_nX1xamM7YfmaxWHscd6djR88HBzBzgCLjEQLOqc_GDrGloS3jqaEMa6Pa7_R3InWNRhP8lYdjVKt5_WmFhmfeq7R9rQsLn804hB34_T1o82bW2GecegBaPF0MeDJvoAOMv-xEqb21cOUC4rBdqo-jDWGu0wrwx0otC0UBDfbRbhsrlKV75Yoe204NouUWSaEjJiMRgzOskPb2fmbDMEkTKAgsfk1J2teFrpXhbgGi4M3kStpRgAG2H8cUhTyE9ubcwcT62oIgvG3Rldfa6y0-5A04HyXR1jtHweJsFCL57zJ4jgRNICEOi6A_l1s16EaUYIk3W0CWSXpuaIE384PUel96bBbc4iO3XRimlOpYSXtorulM795V0zauSrp_wrqUozQ2UOstJMF7o8l204ei39jeFFwpI2BM_xlh54ROWt6mbwwY312qeB50aWz0M3wvxddqu3a_20x-zyokwhpMmh0fbm40IIOaFa-Y1B834BY6aUfyQwSRZq6UfO0kUyoWMah1ywmkqMV9kz75ta6ekSnY0QwroMQYbId31dMOWZuaOcaZV9Ok9OdxapZM8B1d8U-QuXORsJv_fa6IdYUgdjwz4rwFkJiM6_zSfkU200urFzOphvcdVkYoppW7mr67rSBznrwqYI0677FtJUqthfnrtJ_ZJpo-t9ougguffaQipJwjFwZmyfoS1fOubfzqatBVnurW87R2B-JlY997ob3bffMZISuQ27MXt9JM5hfYgbyRns39g57m4v6ktVOBvVq7am8w-wqse7kbi0QIa5JjYBoeetzbMu454uev27UZiumv9kIMVFo0e9kp1V-wfXqIymj-74ybm9JyQ8wmqckcI8s9O9q5hXJ2nop_9iR_tj5QqsBmSnZtlBY57MleSsgy_IXW5ktzjez9Rw2SjakQ_i0xl6pi7wZ3q5mX8o2F45vj2Mo7woJB-BhpVcvScq0Fqbv9iavFF7'
    },
    params: { mimeType: 'application/pdf' },
    validateStatus: [Function (anonymous)],
    retry: true,
    errorRedactor: [Function: defaultErrorRedactor],
    retryConfig: {
      currentRetryAttempt: 0,
      retry: 3,
      httpMethodsToRetry: [Array],
      noResponseRetries: 2,
      statusCodesToRetry: [Array]
    }
  },
  response: {
    config: {
      url: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf',
      method: 'GET',
      responseType: 'stream',
      userAgentDirectives: [Array],
      paramsSerializer: [Function (anonymous)],
      headers: [Object],
      params: [Object],
      validateStatus: [Function (anonymous)],
      retry: true,
      errorRedactor: [Function: defaultErrorRedactor]
    },
    data: '{\n' +
      '  "error": {\n' +
      '    "code": 400,\n' +
      '    "message": "This file is too large to be exported.",\n' +
      '    "errors": [\n' +
      '      {\n' +
      '        "message": "This file is too large to be exported.",\n' +
      '        "domain": "global",\n' +
      '        "reason": "badRequest",\n' +
      '        "location": "export.size",\n' +
      '        "locationType": "other"\n' +
      '      }\n' +
      '    ]\n' +
      '  }\n' +
      '}\n',
    headers: {
      'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
      'cache-control': 'private, max-age=0',
      'content-length': '321',
      'content-type': 'application/json; charset=UTF-8',
      date: 'Wed, 28 Feb 2024 16:28:46 GMT',
      expires: 'Wed, 28 Feb 2024 16:28:46 GMT',
      server: 'ESF',
      vary: 'Origin, X-Origin',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'SAMEORIGIN',
      'x-guploader-uploadid': 'ABPtcPqCOaSGjiUlM-J9pa8kUkAtcLeR4NDexI63N2ecZ7dCEkTryxssJKL7HmpIDi5QJWZyPp3eaFaUew',
      'x-xss-protection': '0'
    },
    status: 400,
    statusText: 'Bad Request',
    request: {
      responseURL: 'https://www.googleapis.com/drive/v2/files/1G11FllbKw6qLbYJM1N8TdZpb6G2Q4QazslxsgLw9oO4/export?mimeType=application%2Fpdf'
    }
  },
  error: undefined,
  status: 400,
  [Symbol(gaxios-gaxios-error)]: '6.3.0'
}

Thanks!

mn-featurewave commented 7 months ago

Any updates on this? This still looks like there and not able to download files larger than 12MB or so.