Mailing List Records

[swflint]

I've started working on the mailing-list record type. I'm going to leave this here, and as a draft, but I'd appreciate your comments (and help)!

[swflint]

Related to #92

[girzel]

Thanks for working on this!

I guess my first instinct was to use further record subclasses to represent different types of mailing list. So ebdb-record-mailing-list would be a parent class, and there would be subclasses for mailman, google, etc, as necessary. In general, in EBDB, I've tried to use classes and subclasses to answer the question "what is it", and fields just to hold instance-level data. Using a list-type field class seems to confuse this separation of concerns: if we want to know exactly how a mailing-list instance should behave, we would need to first look at its class, then look at what sort of list-type field it had.

Practically, this shakes out in the definition of generic functions. If all "identity" information is held in the class, generic functions are written as:

(cl-defgeneric ebdb-mailing-list-do-something ((rec ebdb-mailing-list-mailman))
  (do-something-mailman))

Whereas if we use a "list-type" slot+field, it's:

(cl-defgeneric ebdb-mailing-list-do-something ((rec ebdb-mailing-list))
  (let ((list-type (slot-value rec 'list-type)))
    (cl-case (slot-value list-type 'list-type)
      ("mailman" (do-something-mailman))
      ("google" (do-something google)))))

The former ends up being more code (and it actually runs slower, not that I think that's a big deal), but has the advantage that third-party developers could write an add-on (someday they will! I can dream!) defining a new mailing-list type, without having to touch EBDB at all. In the latter situation, we'd have to let them "get inside" the generic function, which would probably mean a lookup in an alist defvar, the way Elisp usually does it. I've tried to avoid that sort of thing in EBDB.

But it would be interesting to see how far we can get without needing to know about the exact type of mailing list. Specifically, well-formed lists all send the same set of headers: List-ID, List-Archive, List-Help, etc. I think the first thing to do is think about how to store this information (a single field type with separate slots for each header type? separate slots on the record itself?) in a way that will make it easy for the user to, eg, open the archive URL for a mailing list.

In the end we may find that we don't actually need to specify whether the mailing list is Google, mailman, etc.

Thanks again!

[swflint]

Part of the reason I used the type field is that it makes it easy to edit/change (I've had lists switch hosting on me, sadly), though I imagine it wouldn't be bad either way. I also thought of using something like an ebdb-mailing-list-action-alist where action is some mailing list action, and it's organized with types as keys. The use of generics, however, is probably better, as it would let us store, for instance, mailing list type specific information (to do certain actions with listserv, iirc, you need to send a password).

[girzel]

Huh, I hadn't thought of mailing lists changing providers. But I still think there's hope to get by without specifying type at all, until we really confirm that there are lists out there that don't conform to the standard headers.

I guess I'm leaning toward one slot to hold the List-ID, another slot with a field type that holds all the URLs and mail addresses for control (help, unsubscribe, subscribe, post, etc), and has a list of "actions" that let you pick which you're using, and maybe a slot holding an instance of ebdb-field-obfuscated, for the password? Dunno, just spitballing. What else would we want?

[swflint]

That's probably a better option then. I'd still think that maybe a way of handling modifying reply-to would be useful. Keeping List-ID and similar would be helpful. But each type can generally have its control addresses deduced pretty easily, and I'd rather generate that as needed than have to store all of them.

ListServ in particular is a funny one, and part of the reason for the line of subtypes or keeping types thought (you have one address, and you send commands to that, often in a batch). Not to mention its weird attachments feature. As a note, those passwords tend to be short and sent back in plain text on a semi-regular basis.

[girzel]

I agree that generating as much as possible is a good idea.

Maybe we need a bit of taxonomy to see exactly how different the various list servers are from one another?

[swflint]

Agreed.

[swflint]

So, here's what I'm thinking (and whether or not we auto-detect or something else), we use the object-based system, and we split into two main types, address-based control and command-based control. Because, for the most part, auto-detection of addresses is possible, I don't think further specialization is necessary. Command-based control may require it, but even then, I'm not sure.

Also: how can I make the records displayable? Because I don't want them to be entity-types (as there's no need for all that goes along with that), I can't get them to display.

[girzel]

On June 10, 2020 8:56:25 AM PDT, "Samuel W. Flint" notifications@github.com wrote:

So, here's what I'm thinking (and whether or not we auto-detect or something else), we use the object-based system, and we split into two main types, address-based control and command-based control. Because, for the most part, auto-detection of addresses is possible, I don't think further specialization is necessary. Command-based control may require it, but even then, I'm not sure.

Okay, sounds good! Is it possible to use this or another subclass to record NNTP groups in Gnus?

Also: how can I make the records displayable? Because I don't want them to be entity-types (as there's no need for all that goes along with that), I can't get them to display.

Yes, I'll look at dropping the specialization down to ebdb-record, I don't see why not. Or, if we wanted a significantly different display for mailing lists, just write new displays. There's no reason they have to be lists of properties, or look anything like the other records; perhaps they could display as a list of buttons providing mailing list action options. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[swflint]

Eric Abrahamsen writes:

EA> On June 10, 2020 8:56:25 AM PDT, "Samuel W. Flint" <notifications@github.com> wrote:
>> So, here's what I'm thinking (and whether or not we auto-detect
>> or something else), we use the object-based system, and we split
>> into two main types, address-based control and command-based
>> control. Because, for the most part, auto-detection of addresses
>> is possible, I don't think further specialization is
>> necessary. Command-based control may require it, but even then,
>> I'm not sure.

EA> Okay, sounds good! Is it possible to use this or another
EA> subclass to record NNTP groups in Gnus?

I'd add another subclass for that -- which given how it would have to be dealt with would probably make sense.

>> Also: how can I make the records displayable? Because I don't
>> want them to be entity-types (as there's no need for all that
>> goes along with that), I can't get them to display.

EA> Yes, I'll look at dropping the specialization down to
EA> `ebdb-record`, I don't see why not. Or, if we wanted a
EA> significantly different display for mailing lists, just write
EA> new displays. There's no reason they have to be lists of
EA> properties, or look anything like the other records; perhaps
EA> they could display as a list of buttons providing mailing list
EA> action options.

Tbh, I probably missed what I needed to do to make it displayable. I do think it would be helpful to re-think how they're displayed -- but name, primary address, and then potentially some buttons for actions would be a reasonable design.

[girzel]

On 06/11/20 10:30 AM, Samuel W. Flint wrote:

Eric Abrahamsen writes:

EA> On June 10, 2020 8:56:25 AM PDT, "Samuel W. Flint"
EA> <notifications@github.com> wrote:
>> So, here's what I'm thinking (and whether or not we auto-detect
>> or something else), we use the object-based system, and we split
>> into two main types, address-based control and command-based
>> control. Because, for the most part, auto-detection of addresses
>> is possible, I don't think further specialization is
>> necessary. Command-based control may require it, but even then,
>> I'm not sure.

EA> Okay, sounds good! Is it possible to use this or another
EA> subclass to record NNTP groups in Gnus?

I'd add another subclass for that -- which given how it would have to be dealt with would probably make sense.

On second thought, I wonder what kind of functionality that could usefully provide. NNTP is pull-only -- there's no concept of subscription, or delivery -- so maybe there's not a lot that could be done there. To be honest, part of the problem is that I tend to reply to newsgroup posts using "S W" in Gnus, which actually uses the mail interface (copying the original author), rather than the news interface. EBDB keeps asking me about creating a record for, eg, emacs-devel@gnu.org, which is only the mail interface to the NNTP newsgroup. Maybe I should just get myself out of the habit of using "S W", and use "F" instead.

Also: how can I make the records displayable? Because I don't want them to be entity-types (as there's no need for all that goes along with that), I can't get them to display.

EA> Yes, I'll look at dropping the specialization down to
EA> `ebdb-record`, I don't see why not. Or, if we wanted a
EA> significantly different display for mailing lists, just write
EA> new displays. There's no reason they have to be lists of
EA> properties, or look anything like the other records; perhaps
EA> they could display as a list of buttons providing mailing list
EA> action options.

Tbh, I probably missed what I needed to do to make it displayable. I do think it would be helpful to re-think how they're displayed -- but name, primary address, and then potentially some buttons for actions would be a reasonable design.

Looks like all the display routines already specialize on ebdb-record, so that part should already work fine. If you write new field classes for these records, you might want to customize how they're formatted. The base generic function looks like:

(cl-defmethod ebdb-fmt-field ((_fmt ebdb-formatter)
                  (field ebdb-field)
                  _style
                  (_record ebdb-record))
  "The base implementation for FIELD simply returns the value of
  `ebdb-string'."
  (ebdb-string field))

So it's enough to provide an ebdb-string method for fields, at least in the simple case.

[girzel]

You're doing a prototype of this, not me -- right? :)

[swflint]

Eric Abrahamsen writes:

EA> You're doing a prototype of this, not me -- right? :) — You are
EA> receiving this because you authored the thread.  Reply to this
EA> email directly, view it on GitHub, or unsubscribe.

Yep! Sorry -- been busy lately, trying to get caught up with some things. Sam

-- Samuel W. Flint 4096R/FA13D704 (F50D 862B 4F65 5943 A8C2 EF0E 86C9 3E7A FA13 D704) λs.(s s) λs.(s s)

[swflint]

Btw, would you be interested in moving each type of record (other than the most basic), to its own file? I think it would make extension easier, as well as make maintenance long-term easier.

[girzel]

Huh, I hadn't thought of that. I guess I'd be almost more likely to move the fields out into their own file... Maybe both records and fields? It can get a little tricky because class definitions need come before all references, which has resulted in some odd code positioning already.

ebdb.el is getting a little large... I guess this is a good idea, but not a priority?

[ghost]

Hi, just chiming in to ask if there are any updates on this. I'm currently saving mailing lists as person records but it'd be great having first-class support for them.

[girzel]

Hi, just chiming in to ask if there are any updates on this. I'm currently saving mailing lists as person records but it'd be great having first-class support for them.

Hey! Sorry for the very slow reply. I'm still really interested in getting this done, but it's been hard just getting past the requirements stage.

It seems like, however the actual mailing list records end up working, one of the most important aspects of behavior will be automatic creation: the user shouldn't have to manually select a record type. So I think I'll start working on it from that end, instead.

It looks to me like the "List-Id" header is the most reliable way of detecting a mailing list (I mean there's "Precedence: list", but "List-Id" has the actual information we want). So I will alter ebdb-message-headers to add another header class, which will either be called 'list or 'public.

One problem is that things like newsletters often have a (garbage) "List-Id" header, but obviously no "List-Post". So some part of the system will have to grow some tests: if both "List-Id" and "List-Post" are present, make a mailing list. Otherwise, do nothing (though I suppose we could add a newsletter type record, if for no other purpose than to make unsubscribing easier).

I suppose I can implement this much without having to set in stone everything about the mailing list record class – it's not too disruptive to add new slots to an existing class. I think starting off with a "post-address" mail field, an "unsubscribe-address" mail field, a "receipt-address" mail field (for the user's own email), and an "archive-url" URL field should do it.

I guess ebdb-update-records will be the best place to provide different behavior based on the kind of email this is: personal correspondence, mailing list post, newsletter, one-time security code, Slack notification, etc. Personally I don't use automatic record creation simply because there's a minority of emails I receive that are actually from a person I want to add to EBDB. But we could make it smarter about skipping some types of emails. Something interesting could be done about mails where the "From" and "Return-Path" addresses don't match, for instance...

Anyway, I'm thinking out loud. Let me see how far I can get, and please chime in with comments/requests.