Closed reichert621 closed 3 years ago
Interesting. Here's some thoughts.
Gmail seems to have a Push Notification system that could be used instead of polling. The push notification POST body includes a base64-encoded string that parses to:
{"emailAddress": "user@example.com", "historyId": "9876543210"}
where historyId
refers to a new History object, which represents an atomic change to the user's inbox.
History
has this shape (per this link):
{
"id": string,
"messages": [
{
object (Message)
}
],
"messagesAdded": [
{
object (MessageAdded)
}
],
"messagesDeleted": [
{
object (MessageDeleted)
}
],
"labelsAdded": [
{
object (LabelAdded)
}
],
"labelsRemoved": [
{
object (LabelRemoved)
}
]
}
So I think it's straightforward to get the new Message
(from History.messagesAdded
).
A Message
looks like this:
{
"id": string,
"threadId": string,
"labelIds": [
string
],
"snippet": string,
"historyId": string,
"internalDate": string,
"payload": {
object (MessagePart)
},
"sizeEstimate": integer,
"raw": string
}
We'd have to look further into what MessagePart
looks like.
If you're trying to send a reply and want the email to thread, make sure that:
- The Subject headers match
- The References and In-Reply-To headers follow the RFC 2822 standard.
thanks @a8t! super helpful :)
also going to paste in some sample response payloads from the Gmail API here...
/gmail/v1/users/me/messages
%{
"messages" => [
%{"id" => "1783bb273d3ca8ed", "threadId" => "1783bb273d3ca8ed"},
%{"id" => "1783b6533a33f0f1", "threadId" => "1783b6533a33f0f1"},
%{"id" => "1783b5705b99bde2", "threadId" => "1783b5705b99bde2"},
%{"id" => "1783b56c46b4eef6", "threadId" => "1783b56c46b4eef6"},
%{"id" => "1783b45f10c856ad", "threadId" => "1782884988975d0e"},
# etc.
]
}
/gmail/v1/users/me/messages/#{message_id}
%{
"historyId" => "367404",
"id" => "1783bdc5ee6857a2",
"internalDate" => "1615912000000",
"labelIds" => ["SENT"],
"payload" => %{
"body" => %{"size" => 0},
"filename" => "",
"headers" => [
%{"name" => "MIME-Version", "value" => "1.0"},
%{"name" => "Date", "value" => "Tue, 16 Mar 2021 12:26:40 -0400"},
%{
"name" => "References",
"value" => "<CADrf=fmqwxG84EvjcE8rm-asKaQM-hZSm=kYizrvOX1kavyGtg@mail.gmail.com>"
},
%{
"name" => "In-Reply-To",
"value" => "<CADrf=fmqwxG84EvjcE8rm-asKaQM-hZSm=kYizrvOX1kavyGtg@mail.gmail.com>"
},
%{
"name" => "Message-ID",
"value" => "<CAO-SVC=6y8jKsTpVy=783mroF7v+qfX+DvWwL9SA9xL7iAa4Tw@mail.gmail.com>"
},
%{"name" => "Subject", "value" => "Re: Test subject for Gmail API"},
%{"name" => "From", "value" => "Alex Reichert <alex@papercups.io>"},
%{"name" => "To", "value" => "Alex Reichert <reichertjalex@gmail.com>"},
%{
"name" => "Content-Type",
"value" => "multipart/alternative; boundary=\"000000000000d4bab205bda9d4dd\""
}
],
"mimeType" => "multipart/alternative",
"partId" => "",
"parts" => [
%{
"body" => %{
"data" => "TmljZSEgVGhpcyBpcyBhIHRlc3QgcmVwbHkuDQoNCkxldCdzIHNlZSBob3cgaXQgZ29lcy4NCuGQpw0KDQpPbiBUdWUsIE1hciAxNiwgMjAyMSBhdCAxMjoyNiBQTSBBbGV4IFJlaWNoZXJ0IDxyZWljaGVydGphbGV4QGdtYWlsLmNvbT4NCndyb3RlOg0KDQo-IEhleSBBbGV4IQ0KPg0KPiBUaGlzIGlzIGEgdGVzdCBtZXNzYWdlIGZvciB0aGUgR21haWwgQVBJLg0KPg0K",
"size" => 210
},
"filename" => "",
"headers" => [
%{
"name" => "Content-Type",
"value" => "text/plain; charset=\"UTF-8\""
},
%{
"name" => "Content-Transfer-Encoding",
"value" => "quoted-printable"
}
],
"mimeType" => "text/plain",
"partId" => "0"
},
%{
"body" => %{
"data" => "PGRpdiBkaXI9Imx0ciI-TmljZSEgVGhpcyBpcyBhIHRlc3QgcmVwbHkuPGRpdj48YnI-PC9kaXY-PGRpdj5MZXQmIzM5O3Mgc2VlIGhvdyBpdCBnb2VzLjwvZGl2PjwvZGl2PjxkaXYgaHNwYWNlPSJzdHJlYWstcHQtbWFyayIgc3R5bGU9Im1heC1oZWlnaHQ6MXB4Ij48aW1nIGFsdD0iIiBzdHlsZT0id2lkdGg6MHB4O21heC1oZWlnaHQ6MHB4O292ZXJmbG93OmhpZGRlbiIgc3JjPSJodHRwczovL21haWxmb29nYWUuYXBwc3BvdC5jb20vdD9zZW5kZXI9YVlXeGxlRUJ3WVhCbGNtTjFjSE11YVc4JTNEJmFtcDt0eXBlPXplcm9jb250ZW50JmFtcDtndWlkPTEwN2JkMGRkLTdmODYtNGIzZS04MDM4LTdhZDMyNWUwMmYxZCI-PGZvbnQgY29sb3I9IiNmZmZmZmYiIHNpemU9IjEiPuGQpzwvZm9udD48L2Rpdj48YnI-PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiIGNsYXNzPSJnbWFpbF9hdHRyIj5PbiBUdWUsIE1hciAxNiwgMjAyMSBhdCAxMjoyNiBQTSBBbGV4IFJlaWNoZXJ0ICZsdDs8YSBocmVmPSJtYWlsdG86cmVpY2hlcnRqYWxleEBnbWFpbC5jb20iPnJlaWNoZXJ0amFsZXhAZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IGRpcj0ibHRyIj5IZXkgQWxleCE8ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgaXMgYSB0ZXN0IG1lc3NhZ2UgZm9yIHRoZSBHbWFpbCBBUEkuPC9kaXY-PC9kaXY-DQo8L2Jsb2NrcXVvdGU-PC9kaXY-DQo=",
"size" => 845
},
"filename" => "",
"headers" => [
%{"name" => "Content-Type", "value" => "text/html; charset=\"UTF-8\""},
%{
"name" => "Content-Transfer-Encoding",
"value" => "quoted-printable"
}
],
"mimeType" => "text/html",
"partId" => "1"
}
]
},
"sizeEstimate" => 1927,
"snippet" => "Nice! This is a test reply. Let's see how it goes. ᧠On Tue, Mar 16, 2021 at 12:26 PM Alex Reichert <reichertjalex@gmail.com> wrote: Hey Alex! This is a test message for the Gmail API.",
"threadId" => "1783bdc057a1f4c2"
}
/gmail/v1/users/me/threads
%{
"nextPageToken" => "00264963116145399493",
"resultSizeEstimate" => 250,
"threads" => [
%{
"historyId" => "367407",
"id" => "1783bdf49b1f474d",
"snippet" => "[Webinar] Learn about global data protection and deliverability | View online version Hey there, A lot has changed since GDPR came into effect in May 2018. Have you been keeping up with all the updates"
},
%{
"historyId" => "367404",
"id" => "1783bdc057a1f4c2",
"snippet" => "Nice! This is a test reply. Let's see how it goes. ᧠On Tue, Mar 16, 2021 at 12:26 PM Alex Reichert <reichertjalex@gmail.com> wrote: Hey Alex! This is a test message for the Gmail API."
},
%{
"historyId" => "367255",
"id" => "1783bb273d3ca8ed",
"snippet" => "Cruise to buy rival driverless car startup; ElevateBio raises $525M in VC; Sprinklr confidentially files for an IPO; Gatsby gets green light for $10M Read online | Don't want to receive these"
},
# etc.
]
}
/gmail/v1/users/me/threads/#{thread_id}?format=full
%{
"historyId" => "367404",
"id" => "1783bdc057a1f4c2",
"messages" => [
%{
"historyId" => "367378",
"id" => "1783bdc057a1f4c2",
"internalDate" => "1615911941000",
"labelIds" => ["IMPORTANT", "CATEGORY_PERSONAL", "INBOX"],
"payload" => %{
"body" => %{"size" => 0},
"filename" => "",
"headers" => [
%{"name" => "Delivered-To", "value" => "alex@papercups.io"},
%{
"name" => "Received",
"value" => "by 2002:a2e:9694:0:0:0:0:0 with SMTP id q20csp4647726lji; Tue, 16 Mar 2021 09:26:18 -0700 (PDT)"
},
%{
"name" => "X-Received",
"value" => "by 2002:a92:540d:: with SMTP id i13mr4411823ilb.8.1615911978285; Tue, 16 Mar 2021 09:26:18 -0700 (PDT)"
},
%{
"name" => "ARC-Seal",
"value" => "i=1; a=rsa-sha256; t=1615911978; cv=none; d=google.com; s=arc-20160816; b=Qr3NovZVSSOKvZFihmnEZ93EdGgNJAl+S5Rbb2KvOQPKCp5D55+RDZPP59vrIg2GxE aWoPW+lmYxKvldQK3cZqjeJYe/2BY6WNLLGFgr1LMx8E+4V888sVZJ2e0gJvnM4pOK52 5NJBqpMS55UbB/aJAIRYs0cQEca7oTFg8Cy9GzHkOFqHsR7rPLVmzh9co0YdEOFJcF8l wezz5in3yEtXgOCC2C5xfHIfDfkAj127+bxeipf5H9murFrEn3Ee87jS1ET4JD5XtzAk 7iWPmBoXOqcFasT2s+77wu3TF/cPMKSh0NXK3Ibqtt8osWBheZDtbVxQQHa1UnXlEUoQ yxfA=="
},
%{
"name" => "ARC-Message-Signature",
"value" => "i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=to:subject:message-id:date:from:mime-version:dkim-signature; bh=Utx/9TlOmY4X3JEroB0TzviHX6lVHUqAFBQ7/R2R8gY=; b=NW4WLSuqYPUXePK27OEjYN9/+oo7HhChx2ksGdIugMjYBjK1HWMLa+pObBoltCk54n +ZkAvEpw7oJkRyG7sd33ok4IyoANQz5lw30+mWG4KayI3W0V1tZ8IZiQd8+Pxa36oxvs kw2W9Jn4Ck5slbgWEtpfTEU81Zp3cIAon/sKk39tcWR1gceoB7dg3DRXTUYmP0F2nw1Z /Rx4SXjVBQ+8F5wHnpH7LTm1XzfjcHDPCeulk2+ao7sdlmighY2VaQH95lV+Uu0ASc5Q EbyufEWLFUoGOtwZamMudcIL525qm49pk7DVzcNsnJ+/i2dF9i4YdIoU3NYvhDeDoPPU yyhA=="
},
%{
"name" => "ARC-Authentication-Results",
"value" => "i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gpygfwCF; spf=pass (google.com: domain of reichertjalex@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=reichertjalex@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com"
},
%{"name" => "Return-Path", "value" => "<reichertjalex@gmail.com>"},
%{
"name" => "Received",
"value" => "from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id p79sor8092478iod.73.2021.03.16.09.26.18 for <alex@papercups.io> (Google Transport Security); Tue, 16 Mar 2021 09:26:18 -0700 (PDT)"
},
%{
"name" => "Received-SPF",
"value" => "pass (google.com: domain of reichertjalex@gmail.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41;"
},
%{
"name" => "Authentication-Results",
"value" => "mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gpygfwCF; spf=pass (google.com: domain of reichertjalex@gmail.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=reichertjalex@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com"
},
%{
"name" => "DKIM-Signature",
"value" => "v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Utx/9TlOmY4X3JEroB0TzviHX6lVHUqAFBQ7/R2R8gY=; b=gpygfwCFJoVj4sS985jmGwMFJ+DafVa4QFu13DuBWWajhdV9Dp2tZ+H5Nmb7mljtuf gjCANID4oAOVj9LxrWTTN1ieE9wZHRyCPgqll+68BAXyJ2TMudur90pz0e+hIaxuKsf1 QITbrm04eFnyZ50WTfGKeHtK9nbgxV9MOJUiMHO5nBNBdOPo3ESLwx1SZo/heJgD0PBf z8eye+Sm8PYadgwJ229HPjjs5suqV0LBZPl3fpY4uaPioEtudl2gbWYd3NyWPAIjirny +c9/xUzQJ1EWiHN7+G/pQprTSPJjA60xirJuAOhNC2dcvtozyE9caCUnYRIhSUupCpW8 naLA=="
},
%{
"name" => "X-Google-DKIM-Signature",
"value" => "v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Utx/9TlOmY4X3JEroB0TzviHX6lVHUqAFBQ7/R2R8gY=; b=Cm/H4TDPDfjZcJtbJZXHQcM8+tQWr+GJPKK9CEO/teZ+QKqs0I56/tUAGWj0MG4gEy YrpHY/VYR0e/1+9IMjcenNov3z9dqYwuX6DckT2nfoANgrtQWwawNQJETMLmb8ks7vkY 3K66/0jSATmKnDJ60Zk3ilSeiqbQIVQFG8NTMvLjOCLyb1xKW2F9ptn9IlkIKWFN1j3C ol9WFvyZRv5mRkhhb2XNqumvEOJpvKhohLNnV7N/aw/vK13EFafIQyRdRyVgy12R3+tW j1z1s3IJiRRPjUBQNd+l/cPgm9UOSpvtAFToZE7RV+dOt1F5yZo8G/+g1s+99L7cCL7e cw2Q=="
},
%{
"name" => "X-Gm-Message-State",
"value" => "AOAM530BdJj/E5PqZOZ8x0/TEiySEY2EnvUQIQdJq6gynwWFiQ0se/IN iEhSsVTu+Q5RiTd7+/EE6x4gJQX1DVkpnK1yQOYrTGsgqRM="
},
%{
"name" => "X-Google-Smtp-Source",
"value" => "ABdhPJz1jztn29kswW7Uxfe+7KS0Uapms9nz0R0QAKLlGnyv42TYnsXi9vMQfNCRrGzTNPubpXBI5vV4qbum7v/6Cb4="
},
%{
"name" => "X-Received",
"value" => "by 2002:a6b:fd07:: with SMTP id c7mr3985617ioi.198.1615911977550; Tue, 16 Mar 2021 09:26:17 -0700 (PDT)"
},
%{"name" => "MIME-Version", "value" => "1.0"},
%{
"name" => "From",
"value" => "Alex Reichert <reichertjalex@gmail.com>"
},
%{"name" => "Date", "value" => "Tue, 16 Mar 2021 12:25:41 -0400"},
%{
"name" => "Message-ID",
"value" => "<CADrf=fmqwxG84EvjcE8rm-asKaQM-hZSm=kYizrvOX1kavyGtg@mail.gmail.com>"
},
%{"name" => "Subject", "value" => "Test subject for Gmail API"},
%{"name" => "To", "value" => "Alex Reichert <alex@papercups.io>"},
%{
"name" => "Content-Type",
"value" => "multipart/alternative; boundary=\"0000000000006906a505bda9d373\""
}
],
"mimeType" => "multipart/alternative",
"partId" => "",
"parts" => [
%{
"body" => %{
"data" => "SGV5IEFsZXghDQoNClRoaXMgaXMgYSB0ZXN0IG1lc3NhZ2UgZm9yIHRoZSBHbWFpbCBBUEkuDQo=",
"size" => 56
},
"filename" => "",
"headers" => [
%{
"name" => "Content-Type",
"value" => "text/plain; charset=\"UTF-8\""
}
],
"mimeType" => "text/plain",
"partId" => "0"
},
%{
"body" => %{
"data" => "PGRpdiBkaXI9Imx0ciI-SGV5IEFsZXghPGRpdj48YnI-PC9kaXY-PGRpdj5UaGlzIGlzIGEgdGVzdCBtZXNzYWdlIGZvciB0aGUgR21haWwgQVBJLjwvZGl2PjwvZGl2Pg0K",
"size" => 99
},
"filename" => "",
"headers" => [
%{
"name" => "Content-Type",
"value" => "text/html; charset=\"UTF-8\""
}
],
"mimeType" => "text/html",
"partId" => "1"
}
]
},
"sizeEstimate" => 4863,
"snippet" => "Hey Alex! This is a test message for the Gmail API.",
"threadId" => "1783bdc057a1f4c2"
},
%{
"historyId" => "367404",
"id" => "1783bdc5ee6857a2",
"internalDate" => "1615912000000",
"labelIds" => ["SENT"],
"payload" => %{
"body" => %{"size" => 0},
"filename" => "",
"headers" => [
%{"name" => "MIME-Version", "value" => "1.0"},
%{"name" => "Date", "value" => "Tue, 16 Mar 2021 12:26:40 -0400"},
%{
"name" => "References",
"value" => "<CADrf=fmqwxG84EvjcE8rm-asKaQM-hZSm=kYizrvOX1kavyGtg@mail.gmail.com>"
},
%{
"name" => "In-Reply-To",
"value" => "<CADrf=fmqwxG84EvjcE8rm-asKaQM-hZSm=kYizrvOX1kavyGtg@mail.gmail.com>"
},
%{
"name" => "Message-ID",
"value" => "<CAO-SVC=6y8jKsTpVy=783mroF7v+qfX+DvWwL9SA9xL7iAa4Tw@mail.gmail.com>"
},
%{"name" => "Subject", "value" => "Re: Test subject for Gmail API"},
%{"name" => "From", "value" => "Alex Reichert <alex@papercups.io>"},
%{
"name" => "To",
"value" => "Alex Reichert <reichertjalex@gmail.com>"
},
%{
"name" => "Content-Type",
"value" => "multipart/alternative; boundary=\"000000000000d4bab205bda9d4dd\""
}
],
"mimeType" => "multipart/alternative",
"partId" => "",
"parts" => [
%{
"body" => %{
"data" => "TmljZSEgVGhpcyBpcyBhIHRlc3QgcmVwbHkuDQoNCkxldCdzIHNlZSBob3cgaXQgZ29lcy4NCuGQpw0KDQpPbiBUdWUsIE1hciAxNiwgMjAyMSBhdCAxMjoyNiBQTSBBbGV4IFJlaWNoZXJ0IDxyZWljaGVydGphbGV4QGdtYWlsLmNvbT4NCndyb3RlOg0KDQo-IEhleSBBbGV4IQ0KPg0KPiBUaGlzIGlzIGEgdGVzdCBtZXNzYWdlIGZvciB0aGUgR21haWwgQVBJLg0KPg0K",
"size" => 210
},
"filename" => "",
"headers" => [
%{
"name" => "Content-Type",
"value" => "text/plain; charset=\"UTF-8\""
},
%{
"name" => "Content-Transfer-Encoding",
"value" => "quoted-printable"
}
],
"mimeType" => "text/plain",
"partId" => "0"
},
%{
"body" => %{
"data" => "PGRpdiBkaXI9Imx0ciI-TmljZSEgVGhpcyBpcyBhIHRlc3QgcmVwbHkuPGRpdj48YnI-PC9kaXY-PGRpdj5MZXQmIzM5O3Mgc2VlIGhvdyBpdCBnb2VzLjwvZGl2PjwvZGl2PjxkaXYgaHNwYWNlPSJzdHJlYWstcHQtbWFyayIgc3R5bGU9Im1heC1oZWlnaHQ6MXB4Ij48aW1nIGFsdD0iIiBzdHlsZT0id2lkdGg6MHB4O21heC1oZWlnaHQ6MHB4O292ZXJmbG93OmhpZGRlbiIgc3JjPSJodHRwczovL21haWxmb29nYWUuYXBwc3BvdC5jb20vdD9zZW5kZXI9YVlXeGxlRUJ3WVhCbGNtTjFjSE11YVc4JTNEJmFtcDt0eXBlPXplcm9jb250ZW50JmFtcDtndWlkPTEwN2JkMGRkLTdmODYtNGIzZS04MDM4LTdhZDMyNWUwMmYxZCI-PGZvbnQgY29sb3I9IiNmZmZmZmYiIHNpemU9IjEiPuGQpzwvZm9udD48L2Rpdj48YnI-PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiIGNsYXNzPSJnbWFpbF9hdHRyIj5PbiBUdWUsIE1hciAxNiwgMjAyMSBhdCAxMjoyNiBQTSBBbGV4IFJlaWNoZXJ0ICZsdDs8YSBocmVmPSJtYWlsdG86cmVpY2hlcnRqYWxleEBnbWFpbC5jb20iPnJlaWNoZXJ0amFsZXhAZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IGRpcj0ibHRyIj5IZXkgQWxleCE8ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgaXMgYSB0ZXN0IG1lc3NhZ2UgZm9yIHRoZSBHbWFpbCBBUEkuPC9kaXY-PC9kaXY-DQo8L2Jsb2NrcXVvdGU-PC9kaXY-DQo=",
"size" => 845
},
"filename" => "",
"headers" => [
%{
"name" => "Content-Type",
"value" => "text/html; charset=\"UTF-8\""
},
%{
"name" => "Content-Transfer-Encoding",
"value" => "quoted-printable"
}
],
"mimeType" => "text/html",
"partId" => "1"
}
]
},
"sizeEstimate" => 1927,
"snippet" => "Nice! This is a test reply. Let's see how it goes. ᧠On Tue, Mar 16, 2021 at 12:26 PM Alex Reichert <reichertjalex@gmail.com> wrote: Hey Alex! This is a test message for the Gmail API.",
"threadId" => "1783bdc057a1f4c2"
}
]
}
Encoded:
"PGRpdiBkaXI9Imx0ciI-TmljZSEgVGhpcyBpcyBhIHRlc3QgcmVwbHkuPGRpdj48YnI-PC9kaXY-PGRpdj5MZXQmIzM5O3Mgc2VlIGhvdyBpdCBnb2VzLjwvZGl2PjwvZGl2PjxkaXYgaHNwYWNlPSJzdHJlYWstcHQtbWFyayIgc3R5bGU9Im1heC1oZWlnaHQ6MXB4Ij48aW1nIGFsdD0iIiBzdHlsZT0id2lkdGg6MHB4O21heC1oZWlnaHQ6MHB4O292ZXJmbG93OmhpZGRlbiIgc3JjPSJodHRwczovL21haWxmb29nYWUuYXBwc3BvdC5jb20vdD9zZW5kZXI9YVlXeGxlRUJ3WVhCbGNtTjFjSE11YVc4JTNEJmFtcDt0eXBlPXplcm9jb250ZW50JmFtcDtndWlkPTEwN2JkMGRkLTdmODYtNGIzZS04MDM4LTdhZDMyNWUwMmYxZCI-PGZvbnQgY29sb3I9IiNmZmZmZmYiIHNpemU9IjEiPuGQpzwvZm9udD48L2Rpdj48YnI-PGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxkaXYgZGlyPSJsdHIiIGNsYXNzPSJnbWFpbF9hdHRyIj5PbiBUdWUsIE1hciAxNiwgMjAyMSBhdCAxMjoyNiBQTSBBbGV4IFJlaWNoZXJ0ICZsdDs8YSBocmVmPSJtYWlsdG86cmVpY2hlcnRqYWxleEBnbWFpbC5jb20iPnJlaWNoZXJ0amFsZXhAZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PGJyPjwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29saWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IGRpcj0ibHRyIj5IZXkgQWxleCE8ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgaXMgYSB0ZXN0IG1lc3NhZ2UgZm9yIHRoZSBHbWFpbCBBUEkuPC9kaXY-PC9kaXY-DQo8L2Jsb2NrcXVvdGU-PC9kaXY-DQo="
Decoded:
"<div dir=\"ltr\">Nice! This is a test reply.<div><br></div><div>Let's see how it goes.</div></div><div hspace=\"streak-pt-mark\" style=\"max-height:1px\"><img alt=\"\" style=\"width:0px;max-height:0px;overflow:hidden\" src=\"https://mailfoogae.appspot.com/t?sender=aYWxleEBwYXBlcmN1cHMuaW8%3D&type=zerocontent&guid=107bd0dd-7f86-4b3e-8038-7ad325e02f1d\"><font color=\"#ffffff\" size=\"1\">á§</font></div><br><div class=\"gmail_quote\"><div dir=\"ltr\" class=\"gmail_attr\">On Tue, Mar 16, 2021 at 12:26 PM Alex Reichert <<a href=\"mailto:reichertjalex@gmail.com\">reichertjalex@gmail.com</a>> wrote:<br></div><blockquote class=\"gmail_quote\" style=\"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex\"><div dir=\"ltr\">Hey Alex!<div><br></div><div>This is a test message for the Gmail API.</div></div>\r\n</blockquote></div>\r\n"
Here are some gerneral considerations for evaluating whether using IMAP would be a viable alternative to the Gmail API:
IMAP is the standard for retrieving emails from a server and defined in RFC 3501. IMAP is supported by all email providersš.
IMAP has several ways we could use to determine "new" emails
\Recent
: IMAP servers automatically tag all emails that no client has been notified about yet with this flag. This would be reasonable to use if linked inboxes are never accessed by other clients (unlikely)\Seen
this flag is available on all emails that have not been marked as read. Again only useful if the inbox is not used in other clients (so also not useful).UIDNEXT
is a status attribute of mailboxes and represents the next UID (unique identifier) that will be used by the server when the next email arrives. Storing this attribute and using it when fetching emails should reliably yield only emails since the last fetch².
GMailâs historyId
is probably based on or closely tied to IMAP UIDs.
This is probably the way to go.IMAP allows a client to remain connected and immediately receive new messages. This would make an implementation much more complex, however. So I donât think this would make sense for an MVP.
There are at least three ways in which threading could be implemented at the IMAP client level:
THREAD
, RFC 5256). It is supported by the most common FLOSS IMAP servers Dovecot but not by Gmail or MS Exchange (see matrix at imapwiki.org)X-GM-THRID
(see developer.google.com). This might be useful for an MVP supporting just GMail.There are several Elixir and Erlang IMAP client libraries. None of them show any recent activity. However, since the scope of IMAP is limited, it is not to be expected that a reasonably well-functioning implementation would change frequently:
Alternatively, if we end up focusing on polling emails without long-lasting open connection, simply using Hackney et al might be a viable option.
Using IMAP seems like a viable alternative to Gmailâs API. We would need to evaluate whether eximap
can work well within Papercups or whether we should build our own small wrapper for the IMAP protocol.
Building an IMAP integration will be a bit more work than a Gmail API integration but I donât expect it to be significantly more difficult.
š Notable exceptions are hey.com and tutanota.com but they focus on particular aspects of email and are not suited for customer-support emails anyways.
² There is another attribute, UIDVALIDITY
which we should store alongside UIDNEXT
. If UIDVALIDITY
has changed, then the mailbox [has been] deleted and a new mailbox with the same name [has been] created at a later date
- but this is probably an edge case.
@wmnnd I'm thinking for rough next steps...
(we can worry about how we map these to resources in papercups later)
@wmnnd here are some quick thoughts, hopefully this is a bit more concrete! (I'll try to keep adding more details, but feel free to ask any questions or clarifications)
To start off we're just going to poll for new messages (every 5-10 mins?) rather than rely on a "push" notification from a webhook or something like that. This feature will only be used by <10 beta users to start, so let's not worry too much about scalability issues quite yet.
At a high level we want to:
The main operations we'll need to accomplish this:
See pseudocode below for a bit more detail...
Client.get_events_since(authorization, history_id)
The history_id
here simply represents a unique ID of an event. This method should return a list of "events" since that ID.
Ideally this will give us a clear "diff" (i.e. messages added/removed) since the event ID passed in. (Similar to https://developers.google.com/gmail/api/reference/rest/v1/users.history/list)
This method should be run whenever we do a "sync" with an email account (since we'll start off by just polling for updates). We'll check for "messages added" to determine which conversations need to be created/updated in Papercups.
Client.get_thread(authorization, thread_id)
This method should retrieve the entire thread, along with its messages. The thread details should include a list of messages.
Each message should include:
Client.get_message(authorization, message_id)
This method will retrieve a single message by ID. The shape of the message will be the same as above.
Client.list_threads(authorization)
This method handles retrieving the list of email threads in an inbox. Each thread should have an ID, which can be passed in below to retrieve all the details.
This method should ideally retrieve the list sorted by most recent -> oldest threads.
Client.send_message(authorization, payload)
This method should handle sending messages as the authenticated email account. If the appropriate fields are passed into the message payload (e.g. "In-Reply-To" header, etc), then the message should be sent as a reply to its existing thread.
Problem Users want to sync their
support@
email accounts with Papercups.Why?
support@
email from the customer's perspectiveSolution TODO: spec this out some more!
Some thoughts:
slack_conversation_threads
, i.e. agmail_conversation_threads
tableDescribe alternatives you've considered Need to investigate if Mailgun makes it possible to do something like this more easily đ¤ hard to imagine that would be the case though?
Additional context TBD
Testing This will probably require a fair bit of manual testing.
(TODO: list out some QA test case scenarios)
Questions, or need help getting started? Feel free to ask below, or ping us on Slack :)
(You can also check out our CONTRIBUTING.md)