Protocol: Mail: EWS

[benbucksch]

Implement EWS protocol for Microsoft Exchange on-premise and Office365.

Authentication via username and password (on prem) or OAuth2 (Office365).

[benbucksch]

Given that Exchange 2013 is

• EOL
• has unfixed critical security holes
• only ~12% market share (measured in domains, users are probably even less)
• needs various places with specific workarounds in the code

we don't need to support it anymore in Mustang. I.e. Exchange 2016 is the oldest Exchange server that we support. We support Exchange 2016, 2019, 2022 and Office365.

That said, if you would like to add support for Exchange 2013, you're free to do so :-)

[NeilRashbrook]

I've written some proof of concept code which I've pushed as branch ews-poc. It gets as far as listing folders.

Obviously the OAuth2 class needs to go in its own file, but where? I also need a utility class file for stuff such as XML2JSON, EWSError, EWSItemError, and some utility methods that I put on EWSAccount.

To try it out, you'll need to create the account preferences in the developer tools; you'll need values for hostname (e.g. outlook.office365.com) and emailAddress, then username is optional (defaults to emailAddress), password is required for non-Office 365 Exchange but only supported on Office 365 for certain accounts e.g. neil@beoenx.onmicrosoft.com, and userRealname is optional.

If you're using Office 365 with OAuth then you'll get an error Need interactive login, because I support the interactive flag. You'll want to bypass that in some way, e.g. by setting a breakpoint in the debugger and toggling the flag. Note that once you have a refresh token you can go back to noninteractive login.

I added my own code to fetch from arbitrary hosts because I've only just read your comment on the issue #55 regarding fetching

• needs various places with specific workarounds in the code

Not true; Owl only has some workarounds for Exchange 2010 (SP1), and ExQuilla might even work on Exchange 2007.

[benbucksch]

Owl only has some workarounds for Exchange 2010

Oh, great. We can include Exchange 2013, then. But it remains low priority.

I added my own code to fetch from arbitrary hosts

I'm very curious to see that. More options are better.

Could you please push your code to a branch neil/ews ? Commit whenever you have something working, and definitely at the end of every work day. I will review on a PR level (i.e. all commits of the branch combined).

[benbucksch]

Happy to see that you're working on this and making progress! Congrats.

[NeilRashbrook]

Could you please push your code to a branch neil/ews ?

I did say that I'd pushed to branch ews-poc. I also would have appreciated some guidance as to how to break the code up into individual files.

[benbucksch]

branch ews-poc

Merged to master

how to break the code up into individual files

I've split up the files and commited that change.

[NeilRashbrook]

I've been looking at ky and it's somewhat awkward to use compared to axios.

• Ky doesn't support URL encoded post data by default, unless you pass a URLSearchParams object, whereas Axios will URL encode a regular JSON object (which is readily passed over JPC) if you set the content-type header.
• Axios can return the HTTP status and data at the same time, but ky can't conveniently do this. (I'm not sure what kyCreate even does if you don't ask it to decode the data for you; I guess JPC would make you await the status.)
• Axios will automatically decode the response for you by setting the responseType, but ky requires you to call either json or text as appropriate.
• Axios supports basic authentication, but ky requires to to manually construct your Authorization header.

As such, in branch ews-ky I kept my postHTTP method and just adapted it to use ky instead of axios.

• I renamed OAuth2ErrorEWS to OAuth2ErrorMS as there's nothing specific to EWS in it; it would be needed for OAuth for ActiveSync.
• OAuth2ErrorMS was supposed to take data as a parameter (I don't know why I passed it response as neither Owl nor ExQuilla do this) so I fixed that too.

[benbucksch]

Thanks for your detailed report on ky. I'm not set on ky specifically. However, axios has serious problems which prevent it from working on my machine. We can look into other solutions, mid-term. For now, keeping postHTTP is fine.

I renamed OAuth2ErrorEWS to OAuth2ErrorMS

I had the exactly same though, after my commit. Thanks for doing that.

OAuth2ErrorMS was supposed to take data as a parameter (I don't know why I passed it response as neither Owl nor ExQuilla do this) so I fixed that too.

It used that data for the default error message. What do we now get for 404, 401, 403 etc? BTW, on a relatec topic: I see that you pass throwHttpError = false. What happens in case of HTTP errors? I think we'd want them to throw, no?

branch ews-ky

Merged to master

[NeilRashbrook]

The problem is that we can get an OAuth2 error from the interactive login, where we just have the error parameters and no request.

I pass throwHttpError: false because at least in the EWS case (I can't remember what OAuth does, the Microsoft documentation doesn't specify the HTTP status) it will return an error status if something goes wrong but we want to inspect the response to get the error message.

[NeilRashbrook]

I don't know whether it makes a difference, but export http_proxy="" just sets it to the empty string; you may need unset http_proxy instead.

[benbucksch]

Status update:

EMail

• [x] Login
• [x] List folders
• [x] List messages in folder
• [x] Sync folder, all changes since last syncState
• [x] Read email metadata
• [x] Read email body
• [x] Read email attachments
• [x] Send email
• [x] Send email with attachments
• [x] Sent copy
• [x] Push mail, i.e. server notifies client (us) immediately when new mail arrives #73

Contacts

• [x] Read contacts
• [x] Write contacts
• [x] Read groups
• [x] ... with group participants that are not in the address book
• [x] Write groups
• [ ] Query GAL

Calendar

• [x] Read events
• [x] Create events
• [x] Modify events
• [x] Timezones, including Windows timezones
• [ ] Recurring events #69
• [ ] Invitations in emails #68