2.5.11 and 3.4 search madness with squatter

[gbulfon]

On 2.5.11

While searching for any custom header ("X-*") works on folders that have no cyrus.squat file, once the folder is quattered the search returns always 0 messages. Is there any way to configure squatter 2.5.11 to index ANY header?

On 3.4

• if you have squatter disabled on conf, searching for custom headers always work, as any other search but body (correct)
• if you have squatter enabled but no squat file yet, any search returns 0 messages! Which is bad during a live transition! Is there any way to let it use full msg scan in this case until squat files are available?
• if you have squatter enabled and squat files generated, custom headers search doesn't work anymore. Is there any way to configure squatter 3.4 to index ANY header?

[gbulfon]

I found the reason, this is a real bug, on 2.5.11 up to 2.5.17.

Look here:

https://github.com/cyrusimap/cyrus-imapd/blob/cyrus-imapd-2.5.17/imap/imapd.c#L9456

Let's start considering that imapd_in is always returning upper case strings, no matter what was received in the protocol. Let's also consider that squatter function index_search wants only upper case strings, because he's indexing everything in upper case. In the code above, the header value is mantained upper case, while header name is translated in-place via lcase into lower case (to allow for matching for special header like 'subject'), and then passed lower case to squatter.

A fix would be to translate the header name in-place vi ucase into upper case (or do nothing if we assume imapd_in is always returning upper case strings) and then match for special headers like 'SUBJECT', and then send the upper case header name to squatter.

I will dig now 3.4 to see if there are similar problems.