create TMTP architecture doc

[candlerb]

(This is in response to a request for feedback on golang-nuts)

Proposals to replace SMTP are not new. I am old enough to remember a proposal called IM2000 by no less than the legendary Daniel J. Bernstein[^1] (expanded by others). You can Google for the details, but in short, it didn't go anywhere. One repudiation which is still online is here.

For this new proposal, what I'd really like to see is a proper architectural description, not just a bits-and-bytes protocol. Who are the participants? What roles do they take? How do intermittently-connected nodes participate, and nodes with dynamic IPs? What's the lifecycle of a message? How are messages routed? Is the DNS involved, and if so, how? Are messages relayed via multiple nodes? What are the trust relationships? Then it becomes possible to see what the tradeoffs are between this and SMTP - what problems it solves, which ones it doesn't, and what new ones it generates.

The protocol description talks about "clients" and "servers", and that a client must "login or register" to a server, but that's all. Within your own domain of trust, that's clearly fine. What about a server in my organization contacting a server in your organization? How are those relationships set up - manually?? Is there an N^2 full mesh, or do some nodes act as transit nodes, or something else? Or is the proposal implying that every client has to register directly with every server that hosts one of their contacts?! None of this is clear.

In short, how is it proposed that this will work on the Internet at large, for messaging outside your own organization? What's the sequence of events which permits user X at organization A to send a message to user Y at organization B? Nobody wants a new private messaging system[^2].

Currently the only architectural documentation I can find is a list of goals. For example, one of those goals is "Messaging services must be able to control how members identify themselves to other members and prevent members from impersonating others." However I cannot find anywhere that describes how this new mail architecture and protocol proposes to achieve this goal.

Some of the other stated goals are clearly achievable within the current SMTP/IMAP architecture. For example: "Every organization, whether tiny or enormous, needs a members-only messaging service that cannot receive traffic from external or unapproved senders." You can easily configure a mail server that accepts SMTP submissions on port 587 from authenticated clients only, and does not accept incoming mail on port 25. Job done.

MNM also seems to propose writing new E-mail clients from scratch to work with this protocol. It seems to me it would be simpler to support existing IMAP and SMTP-submission protocols, and then you could point any existing client at it. However this is just speculation on my behalf, without knowing what the proposed architecture is for nodes communicating using TMTP.

I'm sorry if this comes across as negative. Maybe from the inside, it's "obvious" to you what the architecture is - but for me, it's not. If you want this proposal to have traction, then I believe that explaining the architecture is a necessary step.

The equivalent for SMTP is RFC2821 sections 2.1, 2.3, 2.4, 3.6-3.8, 5 and 6.

---

[^1] Best known as a cryptographer and mathematician, but also the author of qmail and ezmlm, so he certainly groks how E-mail works.

[^2] xkcd: 927, 1810, 2365.

[networkimprov]

Brian, thanks so much for taking the time to write. You make a very good point; TMTP is under-documented. Your list of Q's about architecture is great!

I concentrated on producing evaluation-ready code and end-user docs (included in the client) instead of architecture docs, since I guessed working code would impress the most people. I'll add a couple paragraphs to the protocol doc to summarize the architecture. But I think what you're suggesting is a whitepaper, and I agree it's necessary; the question is how urgent is it, and how much work. (I'm currently wrestling with the puzzle of generating publicity for a new open source project!)

I'll post my draft for the architecture summary here for your feedback before adding it.

To address one of your key questions, TMTP isn't federated like SMTP or Matrix. A client connects directly (via TLS) to each server where the user might have messages waiting (like HTTP). That means a separate identity at each service. To make contact with someone outside your normal circles (e.g. work, school, interests/hobbies) you'd use a professional organization (e.g. the AMA for doctors) or a marketplace site which verifies members' real-life identities.

If you're interested to try mnm, I'd be happy to provide sign-up info for the test server. Cheers!

[candlerb]

Hmm, so say two businesses want to exchange messages. Then would both the sender(s) and receiver(s) register for accounts on some common, public B2B exchange service? Or would there be some accepted convention, e.g. that the "buyer" registers for an account on the "seller's" server (or vice versa)? Unfortunately, not all B2B relationships fall neatly into a buyer-seller model.

Or: would I register to your server to receive messages from you, and separately you register to my server to receive messages from me? In that case, there's an issue around how the identities are tied together.

Does each mailbox have a globally unique identity, e.g. <identity>@<server>? In order to send a message to that person/role, would I normally register for an account on that particular <server> ? Or would I persuade that person (out-of-band) to register for an account on my server? In the latter case, again I need to tie the two identities together.

I think you know where this is going. If I want to accept incoming messages from people I don't know (e.g. sales enquiries), then it seems I need to allow random users to create accounts on my server. This is no different to having an open SMTP mailbox. Or else, I keep an account on some third-party server that I trust - but then my sales leads are limited to people who are also prepared to create such an account on the same service.

On the other hand, if I want to run my own server but keep it "closed" to general registration, then I will need some way for my friends to create accounts - perhaps by printing one-time codes on business cards that I hand out in person.

I guessed working code would impress the most people

For me, the concept is more important than the code. Code is (comparatively speaking) easy.

Aside: having a server implementation which has a web interface might be a good demo, as people could participate without having to download a client. Of course, it would be unwieldy when you have many accounts on many servers. But allowing a client to proxy through their primary server to other servers would allow Javascript to provide a unified view (even with same-origin policy).

[networkimprov]

The issue here is that when you receive a message, you need to know who has control over the sender's alias (i.e. stated identity). If it's my company, it's trustworthy. If it's a vendor, it may not be. If it's a semi-public site which verifies real-life identities, do you trust the verification? And yes, I need to explain that somewhere! :-)

There isn't a global @domain component to an address. An account keeps the server's hostname & port, and an address-book for that service. Each account has a separate thread-list view in a browser tab (i.e. no common inbox).

Re B2B, typically a company would host a separate service for customers. In some cases they'd also invite suppliers, e.g. to discuss custom products. I don't rule out federation in all cases (you may need to log all correspondence on your vendors' sites). A B2B exchange would be used to initiate a conversation, which could be forwarded either side's service for drill-down. From Why TMTP:

2. Organizations that need to let certain members hear from the general public, or communicate with untrusted (perhaps anonymous) customers, may establish a separate service instance for that purpose.

   Organizations may also join third party messaging services which allow membership for all involved in a specific field.

I'll need to collect real-world case studies and feedback on the draft before finalizing TMTP 1.0.

Re one-time codes, yes account registration will accept credentials which are checked by an authorization service.

[networkimprov]

An FAQ page, which covers architecture briefly, is under development here: https://github.com/mnmnotmail/website/blob/faq-1/faq.md

I plan to add it to the website later today.

[networkimprov]

The new FAQ is posted. https://mnmnotmail.org/faq.html

[Russtopia]

Hi, very interesting project! Since the client and server are written in Go, would it be feasible to coordinate with the Vugu project for a '100% Go' stack?

[networkimprov]

@Russtopia thanks for the idea. Moving to WASM is a big step, which I'm not ready to evaluate yet. If you'd like to discuss this further, please create an issue here: https://github.com/networkimprov/mnm-hammer/issues