tools/rehash broken for unixhierarchysep in 3.0.0 and all newer versions

[elliefm]

(Discovered while investigating #3016 )

In 3.0.0, we started using the new mbname API (which replaced the older mboxname_parts API) for mailbox name handling.

The "fulldirhash: yes" configuration option makes the directory hashing algorithm use the user's full localpart as the input to the config directory hashing algorithm, and not simply the first letter.

The "hashimapspool: yes" configuration option extends this hashing behaviour to the mail spool as well (not just the config directory).

One consequence of the mbname change is that Cyrus now uses the user's real localpart (e.g. jane.smith) as input for calculating directory hashes, whereas previously it used the caret-escaped version (e.g. jane^smith). (Note that this difference only exists where "unixhierarchysep: yes" is in effect; otherwise, localparts cannot contain dots at all.)

tools/rehash is a tool for moving config and spool directories around after a change to the fulldirhash and/or hashimapspool settings. It is referenced in imapd.conf(5) as the correct tool for this purpose. It has not been updated since the mbname introduction, so is currently broken for unixhierarchysep installations.

It is also worth observing that this tool is not installed along with Cyrus; it exists only in the source distribution. Installations based on pre-built packages may not even have this tool available (even though it's referenced in the documentation).

Finally, it appears that this tool can only toggle between the various hashing configurations it knows about. It is not a general-purpose tool for finding mis-hashed directories and putting them back where they're meant to be (which means it's not useful as a recovery or migration tool).

There's a few steps needed here to clean things up:

• [ ] update tools/rehash to unescape the localpart before using it to calculate the hash (must be backported to 3.0 and 3.2)
• [ ] enhance tools/rehash to call into libcyrus (or some other Cyrus tool) to calculate directory hashes, rather than having its own implementation, to make it future-proof against any further hashing algorithm changes (master only)

And a couple of nice-to-have's:

• [ ] generalise this tool so that it can find and repair mis-hashed config/spool directories (master only)
• [ ] make it into an installable tool, so that it's available when the source directory is not, and so that it's always definitely the same version as the installed Cyrus (master only)
• [ ] the "master only" enhancements might be achieved by rewriting it as a feature of reconstruct, rather than as a standalone script

[elliefm]

This might be the right fix for the first stage, but off the top of my head I have no particular insight as to how to test it to verify that it works and doesn't break something else:

https://github.com/cyrusimap/cyrus-imapd/compare/master...elliefm:v33/3019-fix-tools-rehash

Looks like this tool doesn't currently know about the "archivepartition-", "backuppartition-" or "searchpartition-" directories, either. It really is quite stale!