creating Message and Thread Objects

[lukesmurray]

I decided to go back to the survey and try to implement some of the requests that people want in order to determine what kind of methods we want on these objects. As you can see some things look really hard to do but what I'm more interested in is your thoughts on the code blocks marked as threads and messages. These are rough drafts of what the new api could look like. Some of my thoughts are listed in plain text or italics while many of the changes I've proposed are explained in the python comments. The bullet points represent tasks which people wanted to do within Triaging. It may be worth submitting this as a pull request so we can comment on the implementations line by line.

Triaging Emails

• categorize as low emails that have > 10 recipients

should we make priority a unique category which would add and remove flags

current implementation

# if greater than 10 recipients then add low label
#  and remove high and urgent label
if len(get_recipients()) > 10:
    add_labels(["low"])
    remove_labels(["high", "urgent"])

threads and messages

def on_new_message(message):
    if len(message.recipients() > 10):
        # i think we should try to support the most common use cases with flags
        # marking priorities should not involve having to make sure other
        # priorities are not set
        message.add_labels(["low"])
        message.remove_labels(["high", "urgent"])

• Categorize as low emails that have >10 recipients and I haven't read nor replied to last three emails from sender

this is currently impossible in our model

threads and messages

we could create a contacts object and use the contacts object in places like get_recipients and get_sender.

def on_new_message(message):
    sender = message.sender()
    # this action is something people want to do a lot so we should support it
    last_three_messages = sender.recent_messages(3)
    # i create this isread() function because the flag for read or not read is
    # \SEEN which is not very friendly
    if all([not prev_message.isread() for prev_message in last_three_messages]):
        message.add_labels(["low"])
        message.remove_labels(["high", "urgent"])

• if an email requires read receipt they should be marked with deadlines

this is impossible and not simple to implement

• text contains today tonight tomorrow tag as urgent

current implementation

# not the use of get and set is not python, we should be using python properties
content = get_content()
if any(word in content for word in ["today", "tonight", "tomorrow"]):
    message.add_labels(["urgent"])
    message.remove_labels(["low", "high"])

threads and messages

def on_new_message(message):
    content = message.content()
    if any(word in content for word in ["today", "tonight", "tomorrow"]):
        # this doesn't exist currently but i think it should
        # this could maintain the exclusivity of a set of priorities
        message.priority("urgent")

• if im cc instead of to then mark as lower priorities

this brings up the issue of whether get_recipients should include cc and bcc. I think it should but we should have separate methods to, cc, and bcc to access these fields individually.

it's unclear to me if we support this in the current implementation

threads and messages

def on_new_message(message):
    # since message.to() and message.cc() would be lists of contact objects
    # it would be nice to have string comparisons be this easy
    # not sure if that is possible
    my_email = apples@apples.com
    if my_email not in message.to() or my_email in message.cc() or my_email in message.bcc():
        message.priority("low")

• urgent emails from family members

current implementation

recipients = get_recipients()
family = ["dad@apples.com", "mom@apples.com", "brother@apples.com"]
if any(family_member in recipients for family_member in family):
    message.add_labels(["urgent"])
    message.remove_labels(["low", "high"])

threads and messages

the alternative is already covered but it involves getting good interoperability between strings and Contact objects.

• high emails from my contacts

we don't support this currently since we don't have a concept of contacts

even with the concept of a Contact object we don't differentiate between people you know and people you just receive emails from

• move important emails to the top at a certain time of day, or send reminder

seems hard to implement

• emails with urgent in the subject line should be important

already covered how this could be done.

get_subject() => message.subect()

• urgent emails from a sender with whom i've exchanged 10 emails since yesterday

we should check how date is returned, I think it should be returned to the user as a datetime object. It almost certainly should not be returned to the user as a string since that will be hard to deal with.

current implementation

this is either impossible or hard to do in the current implementation it might be possible with searching but would need low level understanding of imap.

threads and messages

def on_new_message(message):
    sender = message.sender()
    last_ten_messages = sender.recent_messages(10)
    if all(message.date().date() == datetime.today().date()):
        message.priority("urgent")

• a lot people want some function like f(email) => deadline or f(email) => requires response.

this seems hard to implement

• low is emails sent to group mail addresses or lots of people

it's unclear to me if we can address what a group mail address is csail-related is a group mail address but thats not obvious just from the email address itself

@soyapark @lukesmurray : We can tell if it's a message from a mailing list by inspecting IMAP message properties. One of the properties is Precedence. Its value is list if it's from a mailing list. There are also List-Id, List-Unsubscribe, List-Archive, List-Post, List-Help, List-Subscribe

We already have examples of this type of issue above

• people want to be able to check if they are bcc or cc

We already have examples of this above.

• urgent is emails from a person, email from same person 3 times in a day, verification emails, emails from a specific domain

maybe our Contact object should have a property domain. Other than that this is covered.

• low is newsletters, forums, social media

This is hard to implement without third party information

• set email notifications to limited times
  • filter some emails to come im any time but the rest come in bulk once or twice a day

I don't know if this is possible

[lukesmurray]

Priority Based Actions

There are a few issues which come up often when dealing with priority based actions. The first is the issue of when priority is assigned to an email and how to assign ordering to the users functions. If we have triggers going to callbacks then we need to have the callback which sets priority happen before a callback which takes an action based on priority. If these callbacks are subscribed to the same trigger for example on_new_message then this ordering is undetermined. The simplest answer to this is to have the user put assigning and taking action in the same callback but what happens when priority changes. This brings up the second issue. Some users want to be able to perform an action based on the state of a message changing. For example, 24 hours after a message is read archive it. We currently don't have a way of determining if the state of a message has changed and its not clear to me that it is easy to track state. Subscribing to these state changes is another issue. Another issue I identified is how we can schedule functions to be called. For example set mode to rest and in one hour set mode to working. This might be able to be done with set_interval but maybe we want to also have a way to say how many times we want the interval to be called.

The last issue is how we want to express sending emails. Sending emails should allow for things like setting to bcc cc fields on the Message object. But setting these fields for received emails doesn't make sense. Furthermore common actions like forwarding a Message should be easy. I outlined some issues with this below.

Notifications

As far as I know I don't think that we can implement or change notifications using our program. Nevertheless these are some common notification use cases.

• no notification for low priority emails, multiple notifications for high priority emails
• no mobile notification unless urgent
• notify high priority emails at time
• urgent notification every 6 hours, no notifications for other emails
• get low priority email in batches
  • this isn't specifically about notifications but I think functionally it is similar.

Sorting

I also don't think that our program can sort emails. Soya you may know better than me. I've listed the sorting desires here

• order emails by priority
• pin urgent emails to the top of the inbox until a reply is sent

Actions

This section covers the variety of actions expressed in the same manner as the triaging emails section.

• move email to folder based on priority

This request is asked for a lot so it should be easy to implement.

current implementation

# note that get_labels
if "low" in get_labels():
    move("low_priority_email_folder")

threads and messages

# note that this could be the callback from a trigger and might need
# a dependency on the function that assigns priorities
def on_new_message(message):
    if "low" in message.labels():
        message.move("low_priority_email_folder")

• get low priority email in batches

I don't think that this is possible in our current implementation without freezing the system. Freezing the system could occur if the user does something like time.sleep(10000) which they may want to do if it is 9am and they want their code to wait until 3pm to send an email. Also our current implementation doesn't have publicly documented methods for dealing with multiple messages simultaneously.

threads and messages

This implementation is very rough and further thought could be put into it to make it cleaner.

# this could be implemented as a callback from our timer
def every_three_hours():
    # use a global mailbox object to access low priority email
    # i also ignore filtering only messages received in last three hours for
    # brevity
    low_priority_messages = mailbox.filter_messages(lambda m: not m.isread() and m.priority() == "low")
    my_email = "apples@apples.com"

    # here we are faced with decision of if we want our message object
    # to be sendable. This would mean properties like to and cc are editable
    # which does not make sense on received messages

    batch_message = Message()
    batch_message.to(my_email)
    batch_message.content('\n'.join([m.content() for m in low_priority_messages]))

• set up a reminder to reply to an email that is high priority after opening/reading the email

for this to work as the user intended we probably need some way of being notified that the state of a message has changed, i.e. that it has been opened or read. It's not clear that we can do that but ignoring that problem we also need to figure a way to schedule events.

this is not possible in our current implementation

• repeatedly deliver high priority emails until a reply is sent

this runs into the same issue as the previous one

• easily set mode so you can do something like, @lunch, @dinner, @rest 1 hour

this could be implemented in code or could be done externally through an app. Through code this could be hacked together by doing something like if subject is youps:setMode(lunch). Having rest for one hour relies on scheduling.

• send emails from a certain sender to someone else

current implementation

if get_sender() == "example@apples.com":
    send("fwd:" + get_subject(), "recipient@apples.com", get_content())

messages and threads

# this is a clean looking implementation but could become confusing if we add
# things like bcc and cc and want to modify the forwarded message
def on_new_message(message):
    if message.sender() == "example@apple.com":
        message.forward("recipient@apples.com")

# this implementation makes doing things like adding bcc and cc easier but
# might not conform to IMAP or inherent standards for forwarded messages
def on_new_message(message):
    if message.sender() == "example@apples.com":
        message_to_forward = Message()
        message_to_forward.content(message.content())
        message_to_forward.to("recipient@apples.com")

• send emails to my trash if they've been in my inbox X hours even if I haven't read them

this also relies on a scheduler @soyapark : this type of mutation observer is actually really tricky! It would be easy if users only manipulate emails from YoUPS but they will also use their own email interface to add label and stuff. One way to keep up with these is frequently going over entire messages to detect any property changes. If there is any change, we can update it with a timestamp. Can you think of any better way? @lukesmurray

[soyapark]

Two proposals for controlling notifications

1. Intercept incoming messages

It works as following:

1. A user needs to set a filter that moves incoming emails except for a message from say, pickup@youps.csail.mit.edu, to a folder. (We can't do this from YoUPS since the filter is not part of IMAP. So we should ask the user to set it their email interface) By doing so, the user wouldn't get any notification.
2. If an incoming message is a message that the user wants to be notified, then we can invoke a notification by sending exact same message from pickup@youps.csail.mit.edu and change our sender name to the original sender (i.e., Actual Sender Name pickup@youps.csail.mit.edu, note that most email interfaces don't show email address, but only sender name. We can also instruct YoUPS users don't worry about the email address). We will also mark "IMAP Reply-To" attribute to the original sender's address so when they hit Reply, it will show the original sender address.

Only one drawback here is getting complicated if the user already has other filters. Either option: 1) if the user prioritizes this filter, their Youps filter will be overwritten to their other filters. 2) if the user prioritizes it last, then our notification feature will only work for messages that are arriving at "INBOX" folder.

2. Marking messages as read as soon as it arrives

A benefit is users dont need to set anything. A drawback is users' might get push notifications since users' interface and YoUPS is noticing message arrival at the same time, it would be late for us to turn off the notification.

[lukesmurray]

First implementation containing both messages and threads in #80