Running at scale...

[frink]

Can this scale to 10's of 1000's of users on a big server? No, probably not. <- FROM HACKER NEWS

+1 I'm interested in this as well...

It looks like you have begun the limiting work you mentioned. I also see the shipping of indexes as the biggest hurdle to overcome. IPFS-Cluster could be used to mitigate the file size of the stored messages. But the trick is getting the INDEX DB up to date with continual small changes.

I feel like solving scalability could create the missing arc for all in one mail hosting solution both for the hobbiest and professionals alike. Once that hurdle is overcome this may be endlessly horizontally scalable with Anycast DNS using BGP on Low-End-Boxes.

Sadly, that is likely going to become a problem for spammers too. If they can create a cluster to send 10M+ email at less than $1000/mo the business value of spam will increase. All of the DNS security in the world does not help if you can register a new domain and redeploy in a day.

Nevertheless, I feel like unless we have a solution that scales to the size of SendGrid there will still be significant vendor lock-in and price fixing that results from the oligopoly. Business people, even more than individuals have a real need to run things securely on premise.

Can't wait to see where this goes!!!

[mjl-]

That HN comment also mentions the missing spam filter and rate limiter for outbound email. At least the rate limiter has already been implemented. Another change recently went in that added an index for faster database lookups related to spam detection, that was a potential relatively big blocker.

The backup MX and database replication is an important one, and I think it'll happen at some point. Since mox is pretty self-contained and in charge of its dependencies, it should be possible to make such modifications. It doesn't have to be a generic synchronization tool. Just something that can replicate changes and communicate securely between primary and backup mx.

I think spammers can already use cheap (open source) software to (attempt to) send lots of emails. But they are being blocked/detected relatively effectively. I hope that is not going to change, not for the worse at least...

To become an alternative to email-sending services, mox would need to grow some features (e.g. webhooks, there's a github issue for that; and a smarter/more efficient queue for sending messages, can be improved over time in small steps, as needed).

It would be interesting to see/test how many users mox could currently handle. Which limits would be reached? Total number of file descriptors for imap "idle" notification connections? Incoming cpu/memory load from delivery attempts by spammers? Memory due to boltdb? Or something else entirely. I'm sure we'll get reports from someone running into a limit sooner or later. (:

[frink]

I'm interested in replacing the services for sure. The webhook approach could be pretty simple to add. (I'm willing to help) However, it may be better to consider embedding this server as a library rather than being stand alone. Golang tends to lend itself well to doing this and still providing an app if you want. Several top Go projects are built this way. For webhooks this seems like a better approach allowing both net/http and gofiber to include this as needed.

For test, let's figure out what tests need to run and build that into a test library that can be built and deployed on the actual servers running this stuff. That way someone can get the real world scenario instead of some contrived test that may not meet their unique server architecture. (I can think of about 50 things in the underlying server that would invalidate any posted benchmark tests...)