search

denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97k stars 5.36k forks source link

base64 decoding with "atob" does not consider web safe base64 encoding in Deno Gmail API Client #19546

Open MichaelHindley opened 1 year ago

MichaelHindley commented 1 year ago

Using https://googleapis.deno.dev/v1/gmail:v1.ts by @lucacasonato , the decodeBase64 function calls the built-in "atob". However, the mail parts in Gmail multipart messages are in a different RFC standard base64 encoding (RFC 4648), so attempting to get messages using this client fails with:

error: Uncaught InvalidCharacterError: Failed to decode base64
const binString = atob(b64);

Normally the workaround would be something like replaceAll("-", "+").replaceAll("_", "/") on the string before passing to atob, but since this is nested deeply in the generated api client, it's not possible.

See https://stackoverflow.com/questions/24812139/base64-decoding-of-mime-email-not-working-gmail-api for other discussion.

I'm not sure if this is the correct place to report this, but the client makes no reference to any issue tracker and I cant find any tests for it, so I went by @lucacasonato as the author and he is a core contributor here.

The message I am testing against, which is a publicly available reply on another repo here on Github, sent as an email. This is the "data" variable that is passed as deserializeMessage(data) in usersMessagesGet.

You can see the base64 data returned from gmail is not compatible with denos atob function that is used in the API Client for gmail.

const data = 'aSBhbSB0b28uIGJ1dCBpIGFtIHVzaW5nIGdvb2dsZSBzaGVldCBPYXV0aC4gSXQncyBub3QgYXV0byByZWZyZXNoIHRva2VuLg0KTm93IGkgYW0gZml4aW5nIGJ5IG1hbnVhbCBmb2xsb3dpbmc6DQpTdGVwIDE6IENsaWNrIHZhcmlhYmxlIGNhdGVnb3J5DQpTdGVwMjogQ2xpY2sgdG8gdG9rZW4gcGF0aA0KU3RlcDM6IFVzaW5nIHNjcmlwdCBjYWxsIHdtaWxsLmdldF92YXJpYWJsZSAodG9rZW5fcGF0aCksIGF0IHRoZSBtb21lbnQgd2luZG1pbGwgaXMgcmV0dXJuaW5nIHRoZSBuZXcgdG9rZW4uIChpZiB5b3UgZG9udCBmb2xsb3cgc3RlcCAxIGFuZCAyLCB0aGUgb2xkIHRva2VuIHdpbGwgYmUgcmV0dXJuDQo8aW1nIHdpZHRoPSIxMzQ1IiBhbHQ9ImltYWd…BzcmM9Imh0dHBzOi8vZ2l0aHViLmNvbS93aW5kbWlsbC1sYWJzL3dpbmRtaWxsL2Fzc2V0cy8xMTAzNDUzODEvYjY5ZTllYzQtNjc5Yy00NjViLWE2ZGEtNDY3NDg3NmE3MzM0Ij4NCg0KDQotLSANClJlcGx5IHRvIHRoaXMgZW1haWwgZGlyZWN0bHkgb3IgdmlldyBpdCBvbiBHaXRIdWI6DQpodHRwczovL2dpdGh1Yi5jb20vd2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMiNpc3N1ZWNvbW1lbnQtMTU5NTgyOTk4MA0KWW91IGFyZSByZWNlaXZpbmcgdGhpcyBiZWNhdXNlIHlvdSBhdXRob3JlZCB0aGUgdGhyZWFkLg0KDQpNZXNzYWdlIElEOiA8d2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMi8xNTk1ODI5OTgwQGdpdGh1Yi5jb20-'

console.log(atob(data))
image

Full data here:

{
  id: "188cab26169338a9",
  threadId: "188c85854acb9b7c",
  labelIds: [
    "IMPORTANT",
    "CATEGORY_PERSONAL",
    "INBOX",
  ],
  snippet: "i am too. but i am using google sheet Oauth. It's not auto refresh token. Now i am fixing by manual following: Step 1: Click variable category Step2: Click to token path Step3: Using script call",
  payload: {
    partId: "",
    mimeType: "multipart/alternative",
    filename: "",
    headers: [
      {
        name: "Delivered-To",
        value: "redaced@example.com",
      },
      {
        name: "Received",
        value: "by 2002:a05:7300:e123:b0:b0:c9ee:1bfb with SMTP id hd35csp2080813dyb;        Sat, 17 Jun 2023 11:51:08 -0700 (PDT)",
      },
      {
        name: "X-Google-Smtp-Source",
        value: "ACHHUZ7x4Ro+4Oe2DBKoG62YD6M/ePPKFhMFOUTVHJfe0gDu+mdIsgCl4ANvt84Ohi4nIJNIn5ku",
      },
      {
        name: "X-Received",
        value: "by 2002:a05:6214:5008:b0:629:e646:bdad with SMTP id jo8-20020a056214500800b00629e646bdadmr7015944qvb.4.1687027867885;        Sat, 17 Jun 2023 11:51:07 -0700 (PDT)",
      },
      {
        name: "ARC-Seal",
        value: "i=1; a=rsa-sha256; t=1687027867; cv=none;        d=google.com; s=arc-20160816;        b=e7IkuFEzSqYrAiI5jD326aCeL6lCjByljjD8e3WSKOZUqw8MtjGcmfTMI9sJLs1TPk         f1P3cq6jJJha5SXzjd4RSL8phrsQvweA+yUzXjfqqbt6gFswqgFC+JQCXQFf1msvO7h9         YMfNLcEwDlM0ZfRMOl45H8ln4OB3yqRgKwFijUlR36eSvElK7OD38ac26IS9Xh4chUs/         wyeofrTuWBK6WO92M/606LVE6jt5yi3KHYj6F6N/8DLBs8T8anqdyTWc7hF/o3KLBXVZ         HagN9uYdHdIUk5hDOyclBfYygzrExAEI95qTrt76a1owDjozzejF6RCL7l9l6E0pmEhE         4WEA==",
      },
      {
        name: "ARC-Message-Signature",
        value: "i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;        h=list-unsubscribe:list-post:list-archive:list-id:precedence         :content-transfer-encoding:mime-version:subject:references         :in-reply-to:message-id:cc:to:reply-to:from:date:dkim-signature;        bh=KWzBKdUYyfEMKj433WuD/8hxMEMQK1fsI2IbzqCqT6g=;        b=W5C5w08PnoNtgTiT8RyUvPfAkHBkfZQZVEdYD32EPE3rWrJY4XfCGb7gClD52H1Ogj         ihmqYTz4wMLGLnTpICyHmkCyOn6YhlMVhf5jLFD5QyD8eEDnJQ+KTo6xrFMBXBcc5ShS         yYWLfP6mrq4uRJxblpLiDPkWziIF0CROmMnmHTIALr3YTSMNiwliYm+mObUK/UUuBjX1         aY9H7STwXgScnbZ4v4ljyfONSNiZzcH8mcBIb9P7gFPtTXK7Yhu+iamFfmyXbIo2Pv01         kDHYVW3Py+7nRSRTbrq3MPUAe2xD00kgnav1sMniv02TQRc/l8ivySb6i95lF1nuKdCg         wMtA==",
      },
      {
        name: "ARC-Authentication-Results",
        value: "i=1; mx.google.com;       dkim=pass header.i=@github.com header.s=pf2023 header.b=eocRnNzr;       spf=pass (google.com: domain of noreply@github.com designates 192.30.252.201 as permitted sender) smtp.mailfrom=noreply@github.com;       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=github.com",
      },
      {
        name: "Return-Path",
        value: "<noreply@github.com>",
      },
      {
        name: "Received",
        value: "from out-18.smtp.github.com (out-18.smtp.github.com. [192.30.252.201])        by mx.google.com with ESMTPS id jp13-20020ad45f8d000000b006261c92791bsi12464080qvb.316.2023.06.17.11.51.07        for <redacted@example.com>        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);        Sat, 17 Jun 2023 11:51:07 -0700 (PDT)",
      },
      {
        name: "Received-SPF",
        value: "pass (google.com: domain of noreply@github.com designates 192.30.252.201 as permitted sender) client-ip=192.30.252.201;",
      },
      {
        name: "Authentication-Results",
        value: "mx.google.com;       dkim=pass header.i=@github.com header.s=pf2023 header.b=eocRnNzr;       spf=pass (google.com: domain of noreply@github.com designates 192.30.252.201 as permitted sender) smtp.mailfrom=noreply@github.com;       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=github.com",
      },
      {
        name: "Received",
        value: "from github-lowworker-7f00a19.ac4-iad.github.net (github-lowworker-7f00a19.ac4-iad.github.net [10.52.113.68]) by smtp.github.com (Postfix) with ESMTP id 29E35940FD9 for <redacted@example.com>; Sat, 17 Jun 2023 11:51:07 -0700 (PDT)",
      },
      {
        name: "DKIM-Signature",
        value: "v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com; s=pf2023; t=1687027867; bh=KWzBKdUYyfEMKj433WuD/8hxMEMQK1fsI2IbzqCqT6g=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID:\t List-Archive:List-Post:List-Unsubscribe:From; b=eocRnNzrEEuaHJBFKfa51fZyMKLf7SzBsFZESFBktjQCgNuJ6JLfmAVLqc6eAQ1il\t HbLwqPiv3pnPm1pH4uinJSAAsbIi8X2/2WRUiBYQDczMpAIbt08q1k4zP6AVheSnwt\t uGnKSyBrfiF+uTO/s00SARMwZSFIujWSSxBcJ3fI=",
      },
      {
        name: "Date",
        value: "Sat, 17 Jun 2023 11:51:07 -0700",
      },
      {
        name: "From",
        value: "royalmaster960 <notifications@github.com>",
      },
      {
        name: "Reply-To",
        value: "\"windmill-labs/windmill\" <reply+AAHYGP5FV7J26A2BNZBA7MGCTMZRXEVBNHHGSAJUC4@reply.github.com>",
      },
      {
        name: "To",
        value: "\"windmill-labs/windmill\" <windmill@noreply.github.com>",
      },
      {
        name: "Cc",
        value: "MichaelHindley <redacted@example.com>, Author <author@noreply.github.com>",
      },
      {
        name: "Message-ID",
        value: "<windmill-labs/windmill/issues/1732/1595829980@github.com>",
      },
      {
        name: "In-Reply-To",
        value: "<windmill-labs/windmill/issues/1732@github.com>",
      },
      {
        name: "References",
        value: "<windmill-labs/windmill/issues/1732@github.com>",
      },
      {
        name: "Subject",
        value: "Re: [windmill-labs/windmill] bug: oauth access token for gmail does not auto-refresh (Issue #1732)",
      },
      {
        name: "Mime-Version",
        value: "1.0",
      },
      {
        name: "Content-Type",
        value: "multipart/alternative; boundary=\"--==_mimepart_648e009b1a394_4cbad034874f6\"; charset=UTF-8",
      },
      {
        name: "Content-Transfer-Encoding",
        value: "7bit",
      },
      {
        name: "Precedence",
        value: "list",
      },
      {
        name: "X-GitHub-Sender",
        value: "royalmaster960",
      },
      {
        name: "X-GitHub-Recipient",
        value: "MichaelHindley",
      },
      {
        name: "X-GitHub-Reason",
        value: "author",
      },
      {
        name: "List-ID",
        value: "windmill-labs/windmill <windmill.windmill-labs.github.com>",
      },
      {
        name: "List-Archive",
        value: "https://github.com/windmill-labs/windmill",
      },
      {
        name: "List-Post",
        value: "<mailto:reply+AAHYGP5FV7J26A2BNZBA7MGCTMZRXEVBNHHGSAJUC4@reply.github.com>",
      },
      {
        name: "List-Unsubscribe",
        value: "<mailto:unsub+AAHYGP5FV7J26A2BNZBA7MGCTMZRXEVBNHHGSAJUC4@reply.github.com>, <https://github.com/notifications/unsubscribe/AAHYGP2JWVQJP3GHWXO5MILXLX4BXANCNFSM6AAAAAAZKB4IWE>",
      },
      {
        name: "X-Auto-Response-Suppress",
        value: "All",
      },
      {
        name: "X-GitHub-Recipient-Address",
        value: "redacted@example.com",
      },
    ],
    body: {
      size: 0,
    },
    parts: [
      {
        partId: "0",
        mimeType: "text/plain",
        filename: "",
        headers: [
          {
            name: "Content-Type",
            value: "text/plain; charset=UTF-8",
          },
          {
            name: "Content-Transfer-Encoding",
            value: "7bit",
          },
        ],
        body: {
          size: 750,
          data: "aSBhbSB0b28uIGJ1dCBpIGFtIHVzaW5nIGdvb2dsZSBzaGVldCBPYXV0aC4gSXQncyBub3QgYXV0byByZWZyZXNoIHRva2VuLg0KTm93IGkgYW0gZml4aW5nIGJ5IG1hbnVhbCBmb2xsb3dpbmc6DQpTdGVwIDE6IENsaWNrIHZhcmlhYmxlIGNhdGVnb3J5DQpTdGVwMjogQ2xpY2sgdG8gdG9rZW4gcGF0aA0KU3RlcDM6IFVzaW5nIHNjcmlwdCBjYWxsIHdtaWxsLmdldF92YXJpYWJsZSAodG9rZW5fcGF0aCksIGF0IHRoZSBtb21lbnQgd2luZG1pbGwgaXMgcmV0dXJuaW5nIHRoZSBuZXcgdG9rZW4uIChpZiB5b3UgZG9udCBmb2xsb3cgc3RlcCAxIGFuZCAyLCB0aGUgb2xkIHRva2VuIHdpbGwgYmUgcmV0dXJuDQo8aW1nIHdpZHRoPSIxMzQ1IiBhbHQ9ImltYWdlIiBzcmM9Imh0dHBzOi8vZ2l0aHViLmNvbS93aW5kbWlsbC1sYWJzL3dpbmRtaWxsL2Fzc2V0cy8xMTAzNDUzODEvYjY5ZTllYzQtNjc5Yy00NjViLWE2ZGEtNDY3NDg3NmE3MzM0Ij4NCg0KDQotLSANClJlcGx5IHRvIHRoaXMgZW1haWwgZGlyZWN0bHkgb3IgdmlldyBpdCBvbiBHaXRIdWI6DQpodHRwczovL2dpdGh1Yi5jb20vd2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMiNpc3N1ZWNvbW1lbnQtMTU5NTgyOTk4MA0KWW91IGFyZSByZWNlaXZpbmcgdGhpcyBiZWNhdXNlIHlvdSBhdXRob3JlZCB0aGUgdGhyZWFkLg0KDQpNZXNzYWdlIElEOiA8d2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMi8xNTk1ODI5OTgwQGdpdGh1Yi5jb20-",
        },
      },
      {
        partId: "1",
        mimeType: "text/html",
        filename: "",
        headers: [
          {
            name: "Content-Type",
            value: "text/html; charset=UTF-8",
          },
          {
            name: "Content-Transfer-Encoding",
            value: "7bit",
          },
        ],
        body: {
          size: 2174,
          data: "PHA-PC9wPg0KPHAgZGlyPSJhdXRvIj5pIGFtIHRvby4gYnV0IGkgYW0gdXNpbmcgZ29vZ2xlIHNoZWV0IE9hdXRoLiBJdCdzIG5vdCBhdXRvIHJlZnJlc2ggdG9rZW4uPGJyPg0KTm93IGkgYW0gZml4aW5nIGJ5IG1hbnVhbCBmb2xsb3dpbmc6PGJyPg0KU3RlcCAxOiBDbGljayB2YXJpYWJsZSBjYXRlZ29yeTxicj4NClN0ZXAyOiBDbGljayB0byB0b2tlbiBwYXRoPGJyPg0KU3RlcDM6IFVzaW5nIHNjcmlwdCBjYWxsIHdtaWxsLmdldF92YXJpYWJsZSAodG9rZW5fcGF0aCksIGF0IHRoZSBtb21lbnQgd2luZG1pbGwgaXMgcmV0dXJuaW5nIHRoZSBuZXcgdG9rZW4uIChpZiB5b3UgZG9udCBmb2xsb3cgc3RlcCAxIGFuZCAyLCB0aGUgb2xkIHRva2VuIHdpbGwgYmUgcmV0dXJuPGJyPg0KPGEgdGFyZ2V0PSJfYmxhbmsiIHJlbD0ibm9vcGVuZXIgbm9yZWZlcnJlciIgaHJlZj0iaHR0cHM6Ly91c2VyLWltYWdlcy5naXRodWJ1c2VyY29udGVudC5jb20vMTEwMzQ1MzgxLzI0NjYyNjgwOS1iNjllOWVjNC02NzljLTQ2NWItYTZkYS00Njc0ODc2YTczMzQucG5nIj48aW1nIHdpZHRoPSIxMzQ1IiBhbHQ9ImltYWdlIiBzcmM9Imh0dHBzOi8vdXNlci1pbWFnZXMuZ2l0aHVidXNlcmNvbnRlbnQuY29tLzExMDM0NTM4MS8yNDY2MjY4MDktYjY5ZTllYzQtNjc5Yy00NjViLWE2ZGEtNDY3NDg3NmE3MzM0LnBuZyIgc3R5bGU9Im1heC13aWR0aDogMTAwJTsiPjwvYT48L3A-DQoNCjxwIHN0eWxlPSJmb250LXNpemU6c21hbGw7LXdlYmtpdC10ZXh0LXNpemUtYWRqdXN0Om5vbmU7Y29sb3I6IzY2NjsiPiZtZGFzaDs8YnIgLz5SZXBseSB0byB0aGlzIGVtYWlsIGRpcmVjdGx5LCA8YSBocmVmPSJodHRwczovL2dpdGh1Yi5jb20vd2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMiNpc3N1ZWNvbW1lbnQtMTU5NTgyOTk4MCI-dmlldyBpdCBvbiBHaXRIdWI8L2E-LCBvciA8YSBocmVmPSJodHRwczovL2dpdGh1Yi5jb20vbm90aWZpY2F0aW9ucy91bnN1YnNjcmliZS1hdXRoL0FBSFlHUFpPWkJNS0xHVUlFVkVHSU9MWExYNEJYQU5DTkZTTTZBQUFBQUFaS0I0SVdFIj51bnN1YnNjcmliZTwvYT4uPGJyIC8-WW91IGFyZSByZWNlaXZpbmcgdGhpcyBiZWNhdXNlIHlvdSBhdXRob3JlZCB0aGUgdGhyZWFkLjxpbWcgc3JjPSJodHRwczovL2dpdGh1Yi5jb20vbm90aWZpY2F0aW9ucy9iZWFjb24vQUFIWUdQNDVTU0ZaS01NQVdQQTQ3U0RYTFg0QlhBNUNORlNNNkFBQUFBQVpLQjRJV0dXR0czM05OVlNXNDVDN09SNFhBWk5NSkZaWEc1TEZJTlhXMjNMRk5aMktVWTNQTlZXV0szVFVMNVVXSlRTN0RaWE5ZLmdpZiIgaGVpZ2h0PSIxIiB3aWR0aD0iMSIgYWx0PSIiIC8-PHNwYW4gc3R5bGU9ImNvbG9yOiB0cmFuc3BhcmVudDsgZm9udC1zaXplOiAwOyBkaXNwbGF5OiBub25lOyB2aXNpYmlsaXR5OiBoaWRkZW47IG92ZXJmbG93OiBoaWRkZW47IG9wYWNpdHk6IDA7IHdpZHRoOiAwOyBoZWlnaHQ6IDA7IG1heC13aWR0aDogMDsgbWF4LWhlaWdodDogMDsgbXNvLWhpZGU6IGFsbCI-TWVzc2FnZSBJRDogPHNwYW4-Jmx0O3dpbmRtaWxsLWxhYnMvd2luZG1pbGwvaXNzdWVzLzE3MzIvMTU5NTgyOTk4MDwvc3Bhbj48c3Bhbj5APC9zcGFuPjxzcGFuPmdpdGh1Yjwvc3Bhbj48c3Bhbj4uPC9zcGFuPjxzcGFuPmNvbSZndDs8L3NwYW4-PC9zcGFuPjwvcD4NCjxzY3JpcHQgdHlwZT0iYXBwbGljYXRpb24vbGQranNvbiI-Ww0Kew0KIkBjb250ZXh0IjogImh0dHA6Ly9zY2hlbWEub3JnIiwNCiJAdHlwZSI6ICJFbWFpbE1lc3NhZ2UiLA0KInBvdGVudGlhbEFjdGlvbiI6IHsNCiJAdHlwZSI6ICJWaWV3QWN0aW9uIiwNCiJ0YXJnZXQiOiAiaHR0cHM6Ly9naXRodWIuY29tL3dpbmRtaWxsLWxhYnMvd2luZG1pbGwvaXNzdWVzLzE3MzIjaXNzdWVjb21tZW50LTE1OTU4Mjk5ODAiLA0KInVybCI6ICJodHRwczovL2dpdGh1Yi5jb20vd2luZG1pbGwtbGFicy93aW5kbWlsbC9pc3N1ZXMvMTczMiNpc3N1ZWNvbW1lbnQtMTU5NTgyOTk4MCIsDQoibmFtZSI6ICJWaWV3IElzc3VlIg0KfSwNCiJkZXNjcmlwdGlvbiI6ICJWaWV3IHRoaXMgSXNzdWUgb24gR2l0SHViIiwNCiJwdWJsaXNoZXIiOiB7DQoiQHR5cGUiOiAiT3JnYW5pemF0aW9uIiwNCiJuYW1lIjogIkdpdEh1YiIsDQoidXJsIjogImh0dHBzOi8vZ2l0aHViLmNvbSINCn0NCn0NCl08L3NjcmlwdD4=",
        },
      },
    ],
  },
  sizeEstimate: 8210,
  historyId: "3289341",
  internalDate: "1687027867000",
}
MichaelHindley commented 1 year ago

Verified this fix works, but is most likely not the prettiest solution:

function decodeBase64(b64: string): Uint8Array {
  const b64Web = b64.replace(/-/g, "+").replace(/_/g, "/");
  const binString = atob(b64Web);
  const size = binString.length;
  const bytes = new Uint8Array(size);
  for (let i = 0; i < size; i++) {
    bytes[i] = binString.charCodeAt(i);
  }
  return bytes;
}

By downloading https://googleapis.deno.dev/v1/gmail:v1.ts locally and referencing https://googleapis.deno.dev/_/base@v1/mod.ts for base.