fastmail / authentication_milter

Email Authentication by SPF/DKIM/DMARC etc.
Other
120 stars 20 forks source link

Issue with Perl dependencies #161

Open patch-work opened 4 hours ago

patch-work commented 4 hours ago
> uname -a
OpenBSD example.com 7.6 GENERIC.MP#338 amd64

> pkg_info | grep p5- | awk '{print $1}'

No perl packages at all. This is a new system.

> doas pkg_add p5-Mail-Milter-Authentication
quirks-7.50 signed on 2024-10-15T23:10:48Z
p5-Mail-Milter-Authenticati...:p5-Email-Date-Format-1.008: ok
p5-Mail-Milter-Authenticati...:p5-Email-Simple-2.216p0: ok
p5-Mail-Milter-Authenticati...:p5-Module-Runtime-0.016p0: ok
p5-Mail-Milter-Authenticati...:p5-Class-Method-Modifiers-2.15: ok
p5-Mail-Milter-Authenticati...:p5-Role-Tiny-2.002004: ok
p5-Mail-Milter-Authenticati...:p5-Sub-Quote-2.006008: ok
p5-Mail-Milter-Authenticati...:p5-strictures-2.000006: ok
p5-Mail-Milter-Authenticati...:p5-Moo-2.005005: ok
p5-Mail-Milter-Authenticati...:p5-MooX-Types-MooseLike-0.29p0: ok
p5-Mail-Milter-Authenticati...:p5-Devel-StackTrace-2.05: ok
p5-Mail-Milter-Authenticati...:p5-Throwable-1.001: ok
p5-Mail-Milter-Authenticati...:p5-Sub-Install-0.929: ok
p5-Mail-Milter-Authenticati...:p5-Params-Util-1.102: ok
p5-Mail-Milter-Authenticati...:p5-Data-OptList-0.114: ok
p5-Mail-Milter-Authenticati...:p5-Sub-Exporter-0.991: ok
p5-Mail-Milter-Authenticati...:p5-Email-Address-XS-1.05: ok
p5-Mail-Milter-Authenticati...:p5-MRO-Compat-0.15: ok
p5-Mail-Milter-Authenticati...:p5-Module-Pluggable-5.2p0: ok
p5-Mail-Milter-Authenticati...:p5-Email-Abstract-3.010: ok
p5-Mail-Milter-Authenticati...:p5-Sub-Name-0.27: ok
p5-Mail-Milter-Authenticati...:p5-Try-Tiny-0.32: ok
p5-Mail-Milter-Authenticati...:p5-Email-Sender-2.601: ok
p5-Mail-Milter-Authenticati...:p5-Email-MIME-Encodings-1.317: ok
p5-Mail-Milter-Authenticati...:p5-Email-MessageID-1.408: ok
p5-Mail-Milter-Authenticati...:p5-MIME-Types-2.26: ok
p5-Mail-Milter-Authenticati...:p5-Text-Unidecode-1.30p0: ok
p5-Mail-Milter-Authenticati...:p5-Email-MIME-ContentType-1.028: ok
p5-Mail-Milter-Authenticati...:p5-Email-MIME-1.954v0: ok
p5-Mail-Milter-Authenticati...:p5-Class-Tiny-1.008: ok
p5-Mail-Milter-Authenticati...:p5-Class-Inspector-1.36p0: ok
p5-Mail-Milter-Authenticati...:p5-File-ShareDir-Install-0.14: ok
p5-Mail-Milter-Authenticati...:p5-File-ShareDir-1.118: ok
p5-Mail-Milter-Authenticati...:p5-Scope-Guard-0.21p0: ok
p5-Mail-Milter-Authenticati...:p5-File-Copy-Recursive-0.45p0: ok
p5-Mail-Milter-Authenticati...:p5-Path-Tiny-0.146: ok
p5-Mail-Milter-Authenticati...:p5-Test-File-ShareDir-1.001002p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-Server-2.014p0: ok
p5-Mail-Milter-Authenticati...:p5-Config-Tiny-2.14p0: ok
p5-Mail-Milter-Authenticati...:p5-Clone-0.46: ok
p5-Mail-Milter-Authenticati...:p5-Math-Base-Convert-0.11p0: ok
p5-Mail-Milter-Authenticati...:p5-SQL-Statement-1.414: ok
p5-Mail-Milter-Authenticati...:p5-FreezeThaw-0.5001p0: ok
p5-Mail-Milter-Authenticati...:p5-MLDBM-2.05p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-Daemon-0.49: ok
p5-Mail-Milter-Authenticati...:p5-PlRPC-0.2020p0: ok
p5-Mail-Milter-Authenticati...:p5-DBI-1.644: ok
p5-Mail-Milter-Authenticati...:p5-DBIx-Simple-1.37p0: ok
p5-Mail-Milter-Authenticati...:p5-Regexp-IPv6-0.03p0: ok
p5-Mail-Milter-Authenticati...:p5-URI-5.28: ok
p5-Mail-Milter-Authenticati...:p5-LWP-MediaTypes-6.02p0: ok
p5-Mail-Milter-Authenticati...:p5-HTTP-Date-6.02p0: ok
p5-Mail-Milter-Authenticati...:p5-IO-HTML-1.004: ok
p5-Mail-Milter-Authenticati...:p5-Encode-Locale-1.05p0: ok
p5-Mail-Milter-Authenticati...:p5-HTTP-Message-6.31: ok
p5-Mail-Milter-Authenticati...:p5-HTML-Tagset-3.24: ok
p5-Mail-Milter-Authenticati...:p5-HTML-Parser-3.83: ok
p5-Mail-Milter-Authenticati...:p5-CGI-4.64: ok
p5-Mail-Milter-Authenticati...:p5-XML-NamespaceSupport-1.12p1: ok
p5-Mail-Milter-Authenticati...:p5-XML-SAX-Base-1.09p0: ok
p5-Mail-Milter-Authenticati...:p5-XML-SAX-1.02p0: ok
p5-Mail-Milter-Authenticati...:p5-XML-LibXML-2.0210v0: ok
p5-Mail-Milter-Authenticati...:p5-HTTP-Cookies-6.10: ok
p5-Mail-Milter-Authenticati...:p5-Net-HTTP-6.19: ok
p5-Mail-Milter-Authenticati...:p5-HTTP-Negotiate-6.01p0: ok
p5-Mail-Milter-Authenticati...:p5-File-Listing-6.15: ok
p5-Mail-Milter-Authenticati...:p5-WWW-RobotRules-6.02p0: ok
p5-Mail-Milter-Authenticati...:p5-HTTP-Daemon-6.16: ok
p5-Mail-Milter-Authenticati...:p5-libwww-6.77: ok
p5-Mail-Milter-Authenticati...:p5-DBD-SQLite-1.74v0: ok
p5-Mail-Milter-Authenticati...:p5-Unicode-Stringprep-1.105p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-IDN-Nameprep-1.102p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-IDN-Encode-2.500p2: ok
p5-Mail-Milter-Authenticati...:p5-Socket6-0.29p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-SSLeay-1.94p0: ok
p5-Mail-Milter-Authenticati...:p5-IO-Socket-SSL-2.089: ok
p5-Mail-Milter-Authenticati...:p5-Digest-HMAC-1.04: ok
p5-Mail-Milter-Authenticati...:p5-Authen-SASL-2.16p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-SMTPS-0.10: ok
p5-Mail-Milter-Authenticati...:p5-Net-IP-1.26p1: ok
p5-Mail-Milter-Authenticati...:p5-Net-DNS-1.46: ok
p5-Mail-Milter-Authenticati...:p5-Crypt-OpenSSL-Bignum-0.09p0: ok
p5-Mail-Milter-Authenticati...:p5-Crypt-OpenSSL-Guess-0.15: ok
p5-Mail-Milter-Authenticati...:p5-Crypt-OpenSSL-Random-0.17: ok
p5-Mail-Milter-Authenticati...:p5-Crypt-OpenSSL-RSA-0.33p0: ok
p5-Mail-Milter-Authenticati...:p5-CryptX-0.081: ok
p5-Mail-Milter-Authenticati...:p5-JSON-4.10: ok
p5-Mail-Milter-Authenticati...:p5-Mail-AuthenticationResults-2.20231031: ok
p5-Mail-Milter-Authenticati...:p5-Module-Implementation-0.09p0: ok
p5-Mail-Milter-Authenticati...:p5-Params-Validate-1.31: ok
p5-Mail-Milter-Authenticati...:p5-Getopt-Long-Descriptive-0.110: ok
p5-Mail-Milter-Authenticati...:p5-Time-TimeDate-2.33: ok
p5-Mail-Milter-Authenticati...:p5-Mail-Tools-2.21p0: ok
p5-Mail-Milter-Authenticati...:p5-Mail-DKIM-1.20240619: ok
p5-Mail-Milter-Authenticati...:p5-Regexp-Common-2024080801: ok
p5-Mail-Milter-Authenticati...:p5-Mail-DMARC-1.20240314: ok
p5-Mail-Milter-Authenticati...:p5-Error-0.17029: ok
p5-Mail-Milter-Authenticati...:p5-NetAddr-IP-4.079p0: ok
p5-Mail-Milter-Authenticati...:p5-Net-DNS-Resolver-Programmable-0.009p0: ok
p5-Mail-Milter-Authenticati...:p5-Mail-SPF-3.20240827: ok
p5-Mail-Milter-Authenticati...:p5-Proc-ProcessTable-0.636: ok
p5-Mail-Milter-Authenticati...:p5-Exporter-Tiny-1.006002: ok
p5-Mail-Milter-Authenticati...:p5-List-MoreUtils-XS-0.430: ok
p5-Mail-Milter-Authenticati...:p5-List-MoreUtils-0.430: ok
p5-Mail-Milter-Authenticati...:p5-Email-Address-1.913: ok
p5-Mail-Milter-Authentication-3.20240701: ok
The following new rcscripts were installed: /etc/rc.d/mail_milter_authentication
See rcctl(8) for details.
New and changed readme(s):
/usr/local/share/doc/pkg-readmes/p5-XML-LibXML

Test

> doas /usr/local/bin/authentication_milter -h
#XS.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)

Possible conflicting files:

> doas find /usr -type f -name XS.pm
/usr/local/libdata/perl5/site_perl/amd64-openbsd/Params/Validate/XS.pm
/usr/local/libdata/perl5/site_perl/amd64-openbsd/Email/Address/XS.pm
/usr/local/libdata/perl5/site_perl/amd64-openbsd/List/MoreUtils/XS.pm
/usr/local/libdata/perl5/site_perl/amd64-openbsd/JSON/XS.pm
/usr/libdata/perl5/ExtUtils/Constant/XS.pm
/usr/libdata/perl5/amd64-openbsd/List/Util/XS.pm
patch-work commented 4 hours ago

The original error message seemed to suggest the existence of a conflict with installed modules. The real meaning was that it did not find the right XS.pm. Infact the perl file /usr/local/bin/authentication_milter includes a call to JSON::XS, and the openbsd package* did not include p5-JSON-XS in the Makefile.

> doas pkg_add p5-JSON-XS
quirks-7.50 signed on 2024-10-15T23:10:48Z
Collision in p5-common-sense-3.75: the following files already exist
/usr/local/libdata/perl5/site_perl/amd64-openbsd/common/sense.pm from p5-common-sense-3.75 (different checksum)
/usr/local/libdata/perl5/site_perl/amd64-openbsd/common/sense.pod from p5-common-sense-3.75 (same checksum)
/usr/local/man/man3p/common::sense.3p from p5-common-sense-3.75 (same checksum)
It seems to be a missing package registration
Repair ? [y/N/a] y
p5-JSON-XS-4.03v1:p5-common-sense-3.75: ok
Collision in p5-Types-Serialiser-1.01: the following files already exist
/usr/local/libdata/perl5/site_perl/Types/Serialiser.pm from p5-Types-Serialiser-1.01 (same checksum)
/usr/local/libdata/perl5/site_perl/Types/Serialiser/Error.pm from p5-Types-Serialiser-1.01 (same checksum)
/usr/local/man/man3p/Types::Serialiser.3p from p5-Types-Serialiser-1.01 (different checksum)
/usr/local/man/man3p/Types::Serialiser::Error.3p from p5-Types-Serialiser-1.01 (different checksum)
It seems to be a missing package registration
Repair ? [y/N/a] y
p5-JSON-XS-4.03v1:p5-Types-Serialiser-1.01: ok
Collision in p5-JSON-XS-4.03v1: the following files already exist
/usr/local/bin/json_xs from p5-JSON-XS-4.03v1 (same checksum)
/usr/local/libdata/perl5/site_perl/amd64-openbsd/JSON/XS.pm from p5-JSON-XS-4.03v1 (same checksum)
/usr/local/libdata/perl5/site_perl/amd64-openbsd/JSON/XS/Boolean.pm from p5-JSON-XS-4.03v1 (same checksum)
/usr/local/libdata/perl5/site_perl/amd64-openbsd/auto/JSON/XS/XS.so from p5-JSON-XS-4.03v1 (different checksum)
/usr/local/man/man1/json_xs.1 from p5-JSON-XS-4.03v1 (different checksum)
/usr/local/man/man3p/JSON::XS.3p from p5-JSON-XS-4.03v1 (different checksum)
/usr/local/man/man3p/JSON::XS::Boolean.3p from p5-JSON-XS-4.03v1 (different checksum)
It seems to be a missing package registration
Repair ? [y/N/a] y
p5-JSON-XS-4.03v1: ok

The module p5-common-sense was not present originally, as there was no module at all, and it is not part of the modules that were installed by p5-Mail-Milter-Authentication, so it was installed by p5-JSON-XS itself.

New test:

> doas /usr/local/bin/authentication_milter -h
Encoder.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)

A new error occurs.

The module Encoder.pm does not occur explicitly in the authentication milter's source code, so is it a resouce internal to the many modules above, or yet another missing module?

> doas find /usr -type f -name Encoder.pm
/usr/local/libdata/perl5/site_perl/amd64-openbsd/Sereal/Encoder.pm
/usr/libdata/perl5/amd64-openbsd/Encode/Encoder.pm
> grep -ai Encoder /usr/local/bin/authentication_milter
[empty]
> grep -ai Encode /usr/local/bin/authentication_milter
print $json->encode( $default_config );
patch-work commented 1 hour ago

On a different system, same OS...

> doas /usr/local/bin/authentication_milter -h
lib/Hash/SharedMem.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)

The file Changes says...

2.20200329.1 2020-03-29 23:55:13+00:00 UTC
- Metrics: Upgrade to Prometheus::Tiny::Shared 0.020
backed by Hash::SharedMem

The module was not installed.

> doas pkg_info | grep -i shared
p5-File-ShareDir-1.118 locate install data per-module
p5-File-ShareDir-Install-0.14 install shared files, to use with File::ShareDir
p5-MaxMind-DB-Common-0.040001p1 code shared by the MaxMind DB reader and writer modules
p5-Test-File-ShareDir-1.001002p0 create fake sharedir for testing
p5-Test-SharedFork-0.35p0 fork test
shared-mime-info-2.4p0 shared mime database for desktops

The module is not available as an OpenBSD package:

https://cdn.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/

Let us use CPAN...

> doas cpan -i Hash::SharedMem
Reading '/root/.cpan/Metadata'
Database was generated on Tue, 18 Jun 2024 07:17:01 GMT
CPAN: HTTP::Tiny loaded ok (v0.086)
CPAN: Net::SSLeay loaded ok (v1.94)
CPAN: IO::Socket::SSL loaded ok (v2.089)
Fetching with HTTP::Tiny:
https://cpan.org/authors/01mailrc.txt.gz
Reading '/root/.cpan/sources/authors/01mailrc.txt.gz'
CPAN: Compress::Zlib loaded ok (v2.204)
............................................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/02packages.details.txt.gz
Reading '/root/.cpan/sources/modules/02packages.details.txt.gz'
Database was generated on Sat, 19 Oct 2024 17:52:40 GMT
CPAN: HTTP::Date loaded ok (v6.02)
..............
New CPAN.pm version (v2.37) available.
[Currently running version is v2.36]
You might want to try
install CPAN
reload cpan
to both upgrade CPAN.pm and run the new version without leaving
the current session.
..............................................................DONE
Fetching with HTTP::Tiny:
https://cpan.org/modules/03modlist.data.gz
Reading '/root/.cpan/sources/modules/03modlist.data.gz'
DONE
Writing /root/.cpan/Metadata
CPAN: Module::CoreList loaded ok (v5.20231129)
Hash::SharedMem is up to date (0.005).
> doas cpan -i Prometheus::Tiny::Shared
Reading '/root/.cpan/Metadata'
Database was generated on Sat, 19 Oct 2024 17:52:40 GMT
CPAN: Module::CoreList loaded ok (v5.20231129)
Prometheus::Tiny::Shared is up to date (0.027).

Did it work?

> doas /usr/local/bin/authentication_milter -h
lib/Hash/SharedMem.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)

No, it did not.

patch-work commented 1 hour ago

Assuming the milter will work at some point, its number of moving parts is so high that a single wheel of this complicated watch will eventually break not only the milter itself but also the mailing system. Will I have the time to fix it, each time it breaks?