Search with accented characters

[berengerzyla]

Hi!

I have an issue with the imap.search when searching in text parts of the message (subject, body..). Let's say I'm looking for Montréal in the subject.

• A quick [ 'ALL', [ 'SUBJECT', 'Montr' ] ] gives me 5 results: 4 - ... voyage Montréal ... 7 - ... voyage Montréal ... 8 - ... voyage Montréal ... 21 - ... voyage Montréal ... 22 - ... voyage Montréal ...

(they're 5 different messages, the numbers are the uids).

• A search with [ 'ALL', [ 'SUBJECT', 'Montréal' ] ] throws an error:

The specified charset is not supported. and textCode: { key: 'BADCHARSET', val: [ 'US-ASCII' ] },

• I'm reacting too quickly being "hey, let's try talking ASCII to him!", so I transform my search (Buffer.from( searchDetails.subject, 'utf8').toString( 'ascii' )), which becomes [ 'ALL', [ 'SUBJECT', 'MontrC)al' ] ]. But of course 0 result, because it's probably interpreted as utf8 C).

What I am missing? How to run a search with special characters?

Reading messages does not show any issue, just the search.

Many thanks!

[mscdex]

AFAIK non-ASCII strings should be automatically detected and handled appropriately. This can be verified by setting debug: console.log and checking that there is a string "literal" instead of a quoted string for the particular string in the search command sent to the server.

The string literal in the debug output will probably look something like this:

{9}
Montréal

[berengerzyla]

...
<= '+ IDLE accepted, awaiting DONE command.'
=> DONE
<= 'IDLE OK IDLE completed.'
=> 'A6 UID SEARCH CHARSET UTF-8 ALL SUBJECT {9}'
<= '+ Ready for additional command text.'
=> 'Montréal\r\n'
<= 'A6 NO [BADCHARSET (US-ASCII)] The specified charset is not supported.'
=> 'IDLE IDLE'

Encoding seems fine when sent. The line-break is not in my original string, I assume the lib does it? This is an Outlook (Office365) server behind that, which seems to only accept US-ASCII, when we send a search with CHARSET UTF-8. Any way to make them talk without conflict? Thanks!

[mscdex]

From my brief google search, it appears Office 365 does not support UTF-8 via IMAP, only US-ASCII, so I'm not sure there's much that can be done client-side. Not supporting UTF-8 in 2020 is absurd, especially for a company/product that large.

[berengerzyla]

OK, and this seems like a final blow to it. https://stackoverflow.com/a/55977511

I'll change my way to go though an API, instead of IMAP. Thanks for checking with me! (and yup, it's a bummer, coming from Microsoft...)