search

cyrusimap / cyrus-imapd

Cyrus IMAP is an email, contacts and calendar server
http://cyrusimap.org
Other
544 stars 148 forks source link

Execute external program from sieve script #3852

Open brianjmurrell opened 2 years ago

brianjmurrell commented 2 years ago

Sometimes one wants to do some kind of processing on incoming messages which exceed Cyrus IMAPD's existing sieve capabilities. Sometimes these are things which are really complicated enough that there is no way they could or should be handled in sieve directly.

For this purpose, it would be nice to be able to pipe a message to an external command.

Dovecot appears to be able to do this. I could not find any information on doing with Cyrus IMAPD sieve though. Am I out of luck? Is this capability a reasonable enhancement request if not currently available?

elliefm commented 2 years ago

Can you give an example of the kind of processing you're talking about? Both in terms of the types of decisions you'd like it to make, and the types of actions you'd like it to take as a result of those decisions.

brianjmurrell commented 2 years ago

I'm not sure I understand the question.

Quite simply I would like one of the "action" options of sieve to be to "Pipe to script" where the path of a script is provided. If the specified match criteria associated with such a rule match, then the mail message is piped to the specified script.

The dovecot implementation shows an example sieve script running a pipe action.

helmut72 commented 2 years ago

For Antispam handling: Dovecot Antispam

elliefm commented 2 years ago

Yes, I understand that you would like a sieve action that pipes the message content to some external script you write.

So, presumably you have specific things in mind that you would like that script to do, once you write it. I'm asking what those things are.

The reason I ask is: maybe those things are possible to achieve from inside Cyrus if we just implement them; or maybe they're impossible to achieve from outside Cyrus, unless we were to also provide an API that your script could call back to.

brianjmurrell commented 2 years ago

Ahh. I see. Well, the sort of things I want to do are never going to be added to the sieve language, which is why I didn't bother to get into details.

I see that @helmut72 provided one kind of example.

There have been numerous times in the past that I have wanted this functionality, but my current specific use case is a script that I wrote that parses the body of a specific recurring e-mail for a date, all using a RE. When the date is extracted using the RE another command is executed to add an entry to a CalDAV calendar.

elliefm commented 2 years ago

Those are helpful examples, thanks.

I had a big writeup here where I was thinking through what kind of shape this might take, but then I had a look at Dovecot's docs and they've already got a specification for a set of sieve extensions to do this and documentation for how to configure and use them in Dovecot, so instead I'll just reference those.

I'll bring this up at the next developers meeting, and we shall see where things go from there.

dilyanpalauzov commented 2 years ago

You can use Sieve to copy the message to a different folder. Then use inotifywait to send each new file from that mailbox to a programme, as described at https://github.com/cyrusimap/cyrus-imapd/issues/3337#issuecomment-787487327 .

Or you could use Sieve to redirect a copy of the message to a different address and the new destination address is prog mailer, which is a script.

helmut72 commented 2 years ago

I don't think users want to do this. They just want to move mails into the spam folder or away, if it's ham.

I've tested something with event notifier:

event_notifier:           external
event_exclude_specialuse: \Drafts \Sent
notify_external:          /var/lib/scripts/notify.sh

But this script will run on every move/copy (vnd.cmu.MessageMove/vnd.cmu.MessageCopy) and not only, when a user move a mail into or away from spam folder. Because it's a shell script, I guess it becomes a performance and maybe a memory hog. Only tested in a lab.

brianjmurrell commented 2 years ago

I'll bring this up at the next developers meeting, and we shall see where things go from there.

Any results from bringing this up?

helmut72 commented 2 years ago

I'll bring this up at the next developers meeting, and we shall see where things go from there.

Any updates? Thanks.

dilyanpalauzov commented 2 years ago

I cannot say what happened at that meeting.

You can use the ideas from https://github.com/cyrusimap/cyrus-imapd/issues/3852#issuecomment-1016279776 to learn/forget messages moved manually to the spam/ham folders.

brianjmurrell commented 2 years ago

I cannot say what happened at that meeting.

Why not?

You can use the ideas from #3852 (comment) to learn/forget messages moved manually to the spam/ham folders.

As has been mentioned before this ticket is NOT just about processing spam/ham. It is about exactly what the title says and that is the ability to execute any external program from a sieve script.

Your solution above only addresses a single use case of this request, which can solve many many other use-cases if implemented.

dilyanpalauzov commented 2 years ago

Why not?

I was not there and I do not know what happened in the meeting.

I have an external notifier, as mentioned above, which handles all uploads and changes (so creating a new file, changing file, deleting file) over CalDAV/CardDAV. You can use the event_notifier/notify_external approach. The Sieve script could also send a copy of the message to a different address, and the recipient of that address might itself be a script.

I also do not understand how can a Sieve script be useful when learing spam/ham. The learning process shall be triggered only on explicit request from the user, or when the anti-spam software scans the messages with an --autolearn option. That said manual moving of a message between IMAP mailboxes does not trigger to my knowledge Sieve script execution.

You are entitled to implement something, which matches your needs, and upload it in the public domain.

I cannot advice you more.

brianjmurrell commented 2 years ago

I was not there and I do not know what happened in the meeting.

So why are you even answering a question about what happened at the meeting then? Your answering about what happened at the meeting implied that you were there. You don't see lots of other people who were not there answering that they don't know what happened either, do you?

I have an external notifier, as mentioned above, which handles all uploads and changes (so creating a new file, changing file, deleting file) over CalDAV/CardDAV. You can use the event_notifier/notify_external approach. The Sieve script could also send a copy of the message to a different address, and the recipient of that address might itself be a script.

Something like that might work, but is nowhere near as straightforward as the Sieve extension being requested in this issue.

brianjmurrell commented 1 year ago

@elliefm said previously over a year ago:

I'll bring this up at the next developers meeting, and we shall see where things go from there.

Did this ever get brought up at any developers meeting(s)? What was the result of the conversations if so? If it didn't, can you please bring it up so that we can get some progress on this?

elliefm commented 1 year ago

I brought it up, though I no longer remember the conversation in much detail, and we either forgot to take minutes or I just can't find them. But there wasn't much enthusiasm for it -- the work to implement it safely and well seemed to far exceed its potential utility (or at least its utility to anyone who attends the meetings), especially considering dilyanpalauzov had already suggested a couple of extant ways to do this sort of thing.

dilyanpalauzov commented 1 year ago

https://www.cyrusimap.org/imap/support/feedback-meetings.html says how to take part in the meetings. Whoever wants can raise this topic there.