Is opendmarc alive?

[gene-git]

There seems to be little to no activity and no release for quite a long time. Is the project alive? If not, what is the alternative (fork, other work etc)? thanks,

[dilyanpalauzov]

The project is not alive, unless somebody contradicts me. Under which criteria will you chose a fork? I have created one at https://mail.aegee.org/cgit/OpenDMARC/.

[gene-git]

Thanks - i am not aware of any other forks. While I can see your repo on my phone, on my desktop (linux w chrome or git clone) i'm unable to see your repo - get protocol error.

[dilyanpalauzov]

I could not get from your answer what you can and what you cannot see.

[gene-git]

On my phone I can use browser to browse your git repo. On desktop, browser fails with SPDY protocol error on the link above. So I then tried git clone https://mail.aegee.org/cgit/OpenDMARC but that failed with unable to access 'https://mail.aegee.org/cgit/OpenDMARC/': HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

Do you have a copy of the repo on github by any chance? I don't have any issues cloning repos on that.

[gene-git]

Forgot to mention - while phone browser lets me see your website - it has warnings that "parts of this site are insecure (such as images). Desktop browser and git command flat out fail for me.

Thank you for reaching back - i'm a bit shocked there is not large interest in a well maintained dmarc milter.

[dilyanpalauzov]

I believe all points are now addressed:

• the page does not download anymore gravatars over HTTP, but uses HTTPS, so that everything is consistently downloaded over HTTPS
• the invalid protocol error for HTTP/2 came, because cgit could not write to its cache, and the error message was sent to over the HTTP/2 stream somehow without necessary wrapping
• git clone https://mail.aegee.org/cgit/OpenDMARC does work now, even when curl uses HTTP/2.

[gene-git]

Yep - It works now - will take a look at what you've done - last commit in the original repo is 6 months ago - having to do with supporting arc - however i don't see any arc headers being added. Question for you - why fork rather than just become an active dev on the original code? Did you try connecting to authors? The reason I ask, is if you do that, then your work will automatically be picked up by the distros as it will in the regular upstream - forking means working with each of the distros to switch over.

Anyway, what are you thinking would be the best way to get your work in all the distros?

thanks for doing this.

gene

[dilyanpalauzov]

The ARC headers are not added by OpenDMARC, but are evaluated, if present, like DKIM-Signature headers are evaluated, when present.

OpenDMARC and OpenDKIM are managed at Github/trusteddomainproject by the same person.

On 20 May 2018 I reported at https://github.com/trusteddomainproject/OpenDKIM/issues/15 suboptimal behaviour with OpenDKIM when used in signed+verify mode and messages were larger than 64kb. This was addressed by the author on 4th November 2018.

On 20 May 2018 I opened https://github.com/trusteddomainproject/OpenDKIM/issues/14 stating, that libunbound is not used correctly. This case was closed by the author, after some changes were considered, but at the end the problem is still there upstream.

On 30 May 2018 I wrote at https://github.com/trusteddomainproject/OpenDMARC/issues/21 that the OpenDMARC code does not compile, it was fixed on 15th November 2018.

So it is not clear, if some problem is submitted, if it will ever be tackled. It might be handled after 6 months, but it might be more and with such uncertainty I do not want to submit information.

I wrote to the author that I do not want to create a fork, as this is just too much energy to convince users to switch to a maintained repository. But at the same time I have done some work on OpenDMARC, and have published this work.

Lets take for example Gentoo as distro and the discussion at https://github.com/trusteddomainproject/OpenDKIM/issues/40. Gentoo uses a tag release, even if the code on master fixes bugs in OpenDKIM ensuring correctness of the DKIM-Signatures. I do not know how to convince that persons of Gentoo to take the code from the develop branch on github and I do not know to convince them to use my fork, instead of the one they currently use. To understand what is going wrong with OpenDKIM 1.10.2 one has to run sufficiently busy email server, evaluate the aggregate reports and see how to get the arrgegare reports zero-free. The maintainers of distributions do not run (sufficiently busy) mailservers on their own, or they do not deploy DMARC on such servers, or they do not evaluate the aggregate reports. So they are not that much in the details to understand the problems OpenDKIM 1.10.2 has and to offer their users better software.

[gene-git]

I wish I could help, but I am just a user, not responsible for maintaining any distro packages (I use Arch).

thanks for your efforts here - perhaps the next step is to ask to become a developer on the trusteddomain source?

[dilyanpalauzov]

Becoming developer on the trusteddomain source, after asking, has not worked.

[AntiFreeze]

@gene-git @dilyanpalauzov - this project is alive. @mskucherawy and myself have been highly focused on progressing ARC through the IETF, so have been more hands off with OpenDMARC than we'd prefer.

Are there some specific issues, PRs, or other concerns that you'd like to see addressed in the near term?

[dilyanpalauzov]

E.g. https://github.com/trusteddomainproject/OpenDMARC/issues/26 is about crashing opendmarc and was filled last Summer.

The question, if the project is alive, arises regularly. If the project were alive, there wouldn’t be such questions.

[gene-git]

Thanks for update - dkim, arc and dmarc are pretty critical currently. I wonder if those 3 projects might not benefit from being merged into a single milter - any thoughts?

thanks you again,

[dilyanpalauzov]

OpenDKIM is complex code. It contains Lua bindings, supports a lot of use cases, integrates libunbound. To get it 100% correct was very, very long process.

It is not yet feature complete. E.g. there is no possibility to add two signatures to a single email. EC signatures with GnuTLS are missing. There are some more small things like stop initializing libunbound on each request.

When you use OpenDMARC, you have a choice between OpenDKIM and dkimpy+dkimpy-milter.

As far as I am concerned, I am very happy to have finally a 100% correct running system, verified by evaluating the aggregate reports, after spending considerable amount of time working on this. I do not want to have yet another challenge merging the three projects into one. Besides, I lost irreversibly my motivation to work on a project, that looks dead.

[AntiFreeze]

@gene-git @dilyanpalauzov thanks for the notes.

A combined milter for spf, dkim, dmarc, and arc is a frequent request. It's definitely on the radar, but there are no concrete plans to build something like that yet - primarily because our top priority is to drive ARC and its supporting needs to completion first.

We do completely agree that some way to make all the milters easy to use together (and to @dilyanpalauzov's point, allow for multiple signatures on a message, too) is critical moving forward, especially as new standards enter the mix. I also personally want better abstractions in the libraries which make them easier to use for non-milter purposes.

[gene-git]

Are there any views on OpenDKIM vs the dkimpy-milter mentioned above? Also, when should we start thinking about switching to EC signing (ed25519) - if we go too soon, things likely to break until there is sufficient number of servers with support for it.

Thanks for the work you all do - much appreciated.

[dilyanpalauzov]

I have no experience with dkimpy-milter.

The next reasonable step towards using EC, is to add two signatures to the outgoing emails from the same domain - one with RSA signature, the other with EC signature.

Nobody knows how long this transition will last, until only EC is usable. But people do not care actually, unless they use DMARC. Offering DMARC is a lot of reading, that few people afford. If Gentoo cared to offer software that signs+verifies correctly, they would have considered, that the “stable“ OpenDKIM version sometimes has defects. To see the defects you need a sufficiently busy mail server and willingness to evaluate the aggregate reports - you can hardly find people going this way.

[mskucherawy]

None of these are abandoned projects. I agree that it takes a long time to get things addressed, and that's unfortunate; the original set of contributors have mostly moved on to other projects, and for the remaining people, available time is very limited. It certainly no longer has full-time support; we rely much more on community support and contributions than was the case previously.

Among other things, the migration from SourceForge to github was somewhat rushed, and we lost reliable access to the OpenDKIM lists; neither of these have been fully resolved.

OpenARC gets most of our attention these days since it's active at the IETF, and is needed to deal with some of the shortfalls of DMARC. That seems like the real pain point of these projects so it gets the lion's share.

I can spend time over this coming weekend resolving issues, especially those with available patches, and cut releases.

[gene-git]

That would be great - thank you for jumping in. These are such core components today, I'm surprised there aren't more folks able and willing to get more involved. I'm happy to build and test but unfortunately not able to work on coding side.

Thanks again.

[mskucherawy]

OpenARC (the newest one, and thus the one getting the most attention lately) is going to be in need of a unit test suite. Contributions there would be great; the unit test suite in OpenDKIM was also a contribution but it's quite old by now and there's probably a newer scheme as a best practice these days.

[mskucherawy]

On the question of merging them into one filter, it has indeed come up before. They were originally implemented independently because you can, for example, run DKIM without SPF or DMARC, but these days they are much more tightly coupled so it makes sense to revisit that decision.

The simplest approach would be to call each of the libraries independently. Although that's cleaner implementation-wise, it's operationally quite expensive.

[gene-git]

Bit that approach is no more expensive than 3 separate milters we have now. So its an improvement in that regard anyway. And if the configs can be brought together via master config which just includes the existing ones we have now that would be a straightforward transition.

[AntiFreeze]

Yes, and from an admin point of view, why manage three or four milters and watch out for proper ordering if that can all be managed properly behind the scenes by a single milter that's easily configurable...

[gene-git]

Curious - are you planning any updates to the git repos of any of openarc, opendmarc or opendkim in the near term?

thanks.

[dilyanpalauzov]

The most common reported error about OpenDKIM is, that it does not canonicalize correctly headers, if there is CRLF immediately after the colon, fixed with

commit a65442161a6e59de49b3db9091754174e363c392
Author: Murray S. Kucherawy <msk@trusteddomain.org>
Date:   Wed Oct 7 00:23:33 2015 -0700

    LIBOPENDKIM: Re-fix bug #226: Deal with header fields that are
    wrapped before there's any content.  Somehow the wrong fix
    was committed.  Originally reported by Alessandro Vesely;
    re-reported by David Stevenson.

diff --git a/libopendkim/dkim-canon.c b/libopendkim/dkim-canon.c
--- a/libopendkim/dkim-canon.c
+++ b/libopendkim/dkim-canon.c
@@ -388,7 +388,7 @@ dkim_canon_header_string(struct dkim_dstring *dstr, dkim_canon_t canon,
                }

                /* skip all spaces before first word */
-               while (*p != '\0' && DKIM_ISWSP(*p))
+               while (*p != '\0' && DKIM_ISLWSP(*p))
                        p++;

                space = FALSE;                          /* just saw a space */

The last release of OpenDKIM is from May 2015. It is rudiculing, that so much time is spent by the users of OpenDKIM reinventing the truth, that version 2.10.1 does not work. In particular, most users never find this out and other problems exists, the users continue using OpenDKIM 2.10.1 and believe they have implemented DKIM/DMARC (correctly). The branch name “develop” is perceived as “do not use”, while the brach name „master” as “do use”.

Relaying on community support means, that the community has to convince the users not to use the master branch/2.10.1 release, but the develop (“do not use”) branch, which convincing does not work, because users cannot or do not want to reproduce a problem caused by 2.10.1, even if they are given detailed instructions, perhaps because they do no know DKIM good enough. Besides, it is not possible to convince all users, to use the develop branch. There is common practice from other software to use only “tagged releases”, which for OpenDKIM means 2.10.1. To sum up, the fact that there is no tagged release containing fixes, costs enormous amount of time of the users and wrong deployments.

The maintainers, or the persons behind the Trusted Domain Project have not made release of OpenDKIM for four years. I asked them to give me access to the repository to make progress and my request was denied. As one of the reasons was given that the Trusted Domain Project has legal obligations. Honestly this is the first time I hear for an open source project, that it denies access, because the project has legal obligations. If the legal obligations are imposed by the Trusted Domain Project and prevent the source code to evolve, then lets dissolve TDP.

To the question, are the projects of TDP alive: If one of these projects has fixed a bug four years ago, and has not made a release since then years, stating repeatedly that the project is alive, cannot be taken seriosly by one, who knows the details.

The common interest is to have correctly implemented DKIM. As a matter of fact, if OpenDKIM has not existed, or was declared “dead with known problems not addressed in a tagged release”, users would have used dkimpy+dkimpy-milter and had probably correct DKIM deployments.

For completeness I add that in November 2018 the code on OpenDKIM has adapted reported fixes, so there is some progress. Estimating however, that in the next four years no tagged release of OpenDKIM will be made, nearly nobody will have any use of that progress.

[gene-git]

Well ... unless something changes soon, I assume the next step is to gather the people willing to work on an alternate system - whether its a fork of this or something else and get the key distro folks involved (fedora, ubuntu, debian, arch ?).

The alternative should have the target state goal of providing a single milter for DKIM, DMARC and ARC.

[gene-git]

Best I can tell, in spite of protests to the contrary, all of the trusteddomainproject's are dead. Since DKIM, DMARC and ARC are critical components I wonder what others are choosing to do.

[dilyanpalauzov]

Before doing any conclusions, it is necessary to wait reasonable time, which is three months for non-urgent matters. Silence of two weeks could mean, that the person was on offline holiday, or in hospital.

In the mean time you could consider taking over the communication work for a fork, that is not connected to programmin.

[iredmail]

@AntiFreeze If no any commit in 2 years and still bugs found and reported in issue tracker, I considered it as dead or abandoned. And no any commit in the OpenARC repo, no even a INSTALL file or tutorial for years. :(

I recently experienced an issue that SPF check passed but opendmarc still rejected it...

[dilyanpalauzov]

The workflow of this project, quite unusual, is to make changes on the develop branch and when there is a release to move them to the master branch. The last change on the develop branch is eight monhts old.

That said, I have little hope, that the maintainers will in long term take care of the TDP projects. They could however help, if they state that under their management the project is dead. This will be a clear statement, making further progress easier.

Honestly, I do not understand what is the point to state, that the project is alive, but then make no releases and so on.

[mskucherawy]

Mr. Palauzov,

I think you have made your point many times. I'm sorry that the project's management does not meet your expectations given our current human resource constraints. It's also unfortunate that you are unwilling to accept the explanations you were given for why it's managed the way it is.

TDP is a corporation that owns intellectual property. Dissolving it is not a trivial matter.

Our choice of license permits you to create your own branch of the project if you intend to take on the work of managing development of the code, documentation, user support, etc., if you wish to continue proclaiming the projects as "dead".

[mskucherawy]

@iredmail: We considered OpenARC to be not quite "finished" because the protocol was still evolving as its defining document underwent revisions. The time we have was spent focused on the specification rather than the code.

ARC was finally published only this month as RFC8617, so now we're in a position that we can reasonably cut a release of it once we address the remaining open issues.

The IETF working group driving ARC will next undertake the work of updating the DMARC specification, which will then lead to work on OpenDMARC to match those changes, or conduct experiments that support that work.

[gene-git]

Thank you for update. Good luck!!

[gene-git]

Hey are you guys (@dilyanpalauzov ) familiar with rspamd? Seems to be a single milter which handles dkim, spf, dmarc and arc as well as a few other things.

It seems to be quite active and actively maintained and may make a good alternative to the 3 openXXX milters - and appear to be standard packages available on Arch anyway.

Wonder if any of you have any experience with rspamd if it might be a worthwhile replacement for the spf, dkim, dmarc and arc?

[rseichter]

My personal opinion is that rspamd suffers from feature creep (IMO it should stick to what is both its strength and original purpose: spam detection). This is a concern to me, not only because most of the workload is tied to a single person (Vsevolod Stakhov). More features mean more complexity and thus a higher risk of bugs, and with only the latest rspamd release being supported, that's a bottleneck.

[gene-git]

Thanks for your thoughts ...

[gene-git]

My concern is that I have opendkim issues with some MS exchange servers where opendkim fails with some large enterprise emails - gmail accepts them fine - since dmarc policy is reject, their mail is being lost due to opendkim - and I need to find a pragmatic path forward that does't lose email as is happening now. The error is that opendkim 'failed to parse Authentication-Results' .. consequence is dmarc reject.

So - in practical terms - does rspamd work "better" than openxxx ...

[gene-git]

Reported this to opendkim issues ... who knows maybe someone has a fix. https://github.com/trusteddomainproject/OpenDKIM/issues/48

[dilyanpalauzov]

Do you use the latest code from git/develop?

[gene-git]

Yes - for dkim its 2.11.0-Beta2 - but that is the same git version as develop arc and dmarc are both git devel directly.