foxcpp / maddy

✉️ Composable all-in-one mail server.
https://maddy.email
GNU General Public License v3.0
5.09k stars 245 forks source link

Dynamic message blob path support #520

Open biancalana opened 2 years ago

biancalana commented 2 years ago

Use case

I'm evaluating maddy for my personal mailserver. I've loved the simplicity and great number of features related to a modern mailserver setup ! Even for personal mailserver its common to have thousand of emails on each mailbox. In my case I have a quite long historic os my mail (almost 20 years) so it's common to have thousand of emails on a folder.

What problem you are trying to solve?

My primary reason to have my own mailserver is privacy thinking that way the considered setup store all the information on my infrastructure. I've chosen to use imap storage blob and discovered that all msg files no matter the account, domain or imap folder are stored on the same directory. My thought is that it will lead do performance problems because I easily going to have thousand of files on the same directory it's not good for any filesystem. Additionally, in system administration perspective, not having a way to split messages by domain and/or accounts will make tasks like copying, moving and managing this harder.

Note alternatives you considered and why they are not useful.

Your idea for a solution

Having config options to allow split messages in different directories for domains, accounts and even imap folders would be great.

How your solution would work in general? We can have a new config option that allow some kind of format specifications (printf % like) to define the way the message path would be expanded. For example: msg_path_format: %d/%a/%f where: d -> domain name a -> email account name f -> imap folder name

I've started to look at the code and still not sure that all these options are possible, but I would like to discuss and willing to do the PR :-)

foxcpp commented 2 years ago

I've started to look at the code and still not sure that all these options are possible, but I would like to discuss and willing to do the PR :-)

With the current maddy design, it would not be easily feasible. The only way I see is to create a way to pass a map of placeholders via context.Context. And I agree that it would be useful and should be implemented.

My thought is that it will lead do performance problems because I easily going to have thousand of files on the same directory it's not good for any filesystem

I believe the performance impact would not be significant for FS implementations based on B-tree structure or similiar. This includes, for example, ext4 which is the most common FS.

Though probably benchmarking would be better.

Chris-Scot commented 1 year ago

Morning Team.

If purely for file system performance rather than organisational reasons, then could the mails be split into subdirectories based on the first 2 chars of the file name?

Personally, I would like to see the mails of each account in separate directories.