userjack6880 / Open-Report-Parser

A Perl based tool to parse DMARC reports from an IMAP mailbox or from the filesystem, and insert the information into a database. Derived from Techsneeze's dmarcts-report-parser
GNU General Public License v3.0
34 stars 10 forks source link

[Request]: PostgreSQL Validation #8

Closed userjack6880 closed 1 year ago

userjack6880 commented 1 year ago

Describe the enhancement request

PostgreSQL functionality is assumed, but needs to be tested.

sporkman commented 1 year ago

Here's a simple one - the $dbtype parameter gets passed to the initial db connect, but the value "postgres" isn't correct for that, it needs to be "Pg".

Working (hard-coding the db type):

my $dbh = DBI->connect("DBI:Pg:database=$dbname;host=$dbhost;port=$dbport", $dbuser, $dbpass)

Fails (with 'install_driver(postgres) failed: Can't locate DBD/postgres.pm in @INC' because the DBI package goes looking for "postgres.pm" when it needs "Pg.pm"):

my $dbh = DBI->connect("DBI:$dbtype:database=$dbname;host=$dbhost;port=$dbport", $dbuser, $dbpass)

I'm also seeing these two index creation lines fail on the tlsrecord table:

dmarc=# CREATE INDEX tlsrecord_idx_serial ON tlsrecord (serial, ip);
ERROR:  column "ip" does not exist
dmarc=# CREATE INDEX tlsrecord_idx_serial ON tlsrecord (serial, ip6);
ERROR:  column "ip6" does not exist
dmarc=#

I'm running through an install now, let me know if there's anything else I can do to check out PG support.

userjack6880 commented 1 year ago

I've just pushed dev code that I've been working on to get this fix out the door. Pull the new code and give it a try.

sporkman commented 1 year ago

I'm testing this out at the moment, just out of the box, it's not creating the db tables. I've verified the login credentials (I'm supplying user/pass/host/port/db) and logged-in as the reports user without issue. When it hits "checkDatabase($dbh);" on line 373 it does not appear to actually do anything, or if it is, it's not outputting any info, even with debug enabled. I'll dig some more.

sporkman commented 1 year ago

Found it (I'd push a fix, but I'm not good at git):

This starts at line 1695:

 # Create missing tables and missing columns.
  for my $table ( keys %{$tables} ) {
    if ($imapauth eq 'simple') {
      next;
    }

    if (!db_tbl_exists($dbh, $table)) {

I think that "imapauth" check there must be a mis-paste or something, as it interrupts the table existence checks if not using OAuth.

userjack6880 commented 1 year ago

I appreciate the feedback - I'll keep these fixes in mind for the next release - assuming you're using a version 2 alpha, considering you mentioned oauth?

userjack6880 commented 1 year ago

Ignore my version comment - for some reason I was thinking this was another project. I've pushed an update that fixes this issue - the three lines were in error. See if the fix helped.

userjack6880 commented 1 year ago

Assuming this issue was fixed.