Undefined variables - Githubissues

goose-ws commented 2 years ago

I checked out previous issues #9 and #15 as well as pull request #12 -- however, I'm not able to get around this error.

Here's my compose file:

version: '3.6'

services:
  dmarc-report:
    image: gutmensch/dmarc-report:1.1
    hostname: dmarc-report
    container_name: dmarc-report
    depends_on:
      - mariadb
    # ports:
      # - "80:80"
    environment:
      - "REPORT_DB_HOST=mariadb"
      - "REPORT_DB_PORT=3306"
      - "REPORT_DB_NAME=dmarc_report"
      - "REPORT_DB_USER=dmarc_report"
      - "REPORT_DB_PASS=[db_pass]"
      - "PARSER_IMAP_SERVER=mail.[domain.tld]"
      - "PARSER_IMAP_PORT=143"
      - "PARSER_IMAP_USER=dmarc@[domain.tld]"
      - "PARSER_IMAP_PASS=[imap_pass]"
      - "PARSER_IMAP_READ_FOLDER=Inbox"
      - "PARSER_IMAP_MOVE_FOLDER=Processed"
      - "PARSER_IMAP_MOVE_FOLDER_ERR=Error"
      - "PARSER_IMAP_IGNORE_ERROR=1"
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
    restart: unless-stopped
    logging:
      driver: json-file
      options: 
        max-file: "3"
        max-size: "10M"

  mariadb:
    image: mariadb:10
    hostname: mariadb
    container_name: mariadb
    command: --skip-innodb-read-only-compressed
    environment:
      - "MYSQL_ROOT_PASSWORD=[db_root_pass]"
      - "MYSQL_DATABASE=dmarc_report"
      - "MYSQL_USER=dmarc_report"
      - "MYSQL_PASSWORD=[db_pass]"
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
      - "/docker/config/mariadb:/var/lib/mysql"
    restart: unless-stopped
    logging:
      driver: json-file
      options: 
        max-file: "3"
        max-size: "10M"

The port is commented out as I'm reverse proxying to the container, rather than accessing the container on its own.

When I pull up the web interface, I get:

Warning: Undefined variable $domains in /var/www/viewer/dmarcts-report-viewer.php on line 315

Warning: Undefined variable $orgs in /var/www/viewer/dmarcts-report-viewer.php on line 316

Warning: Undefined variable $periods in /var/www/viewer/dmarcts-report-viewer.php on line 317

Fatal error: Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/viewer/dmarcts-report-viewer.php:125 Stack trace: #0 /var/www/viewer/dmarcts-report-viewer.php(317): html() #1 {main} thrown in /var/www/viewer/dmarcts-report-viewer.php on line 125

The nginx logs from the container itself confirm this:

[goose@Vergil: ~ ] $ docker logs dmarc-report
2022-03-21 20:44:02,824 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2022-03-21 20:44:02,824 INFO Included extra file "/etc/supervisor/conf.d/crond.conf" during parsing
2022-03-21 20:44:02,824 INFO Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2022-03-21 20:44:02,833 INFO RPC interface 'supervisor' initialized
2022-03-21 20:44:02,833 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2022-03-21 20:44:02,833 INFO supervisord started with pid 19
2022-03-21 20:44:03,837 INFO spawned: 'cron' with pid 21
2022-03-21 20:44:03,839 INFO spawned: 'nginx' with pid 22
2022-03-21 20:44:03,841 INFO spawned: 'php-fpm' with pid 23
[21-Mar-2022 20:44:03] NOTICE: fpm is running, pid 23
[21-Mar-2022 20:44:03] NOTICE: ready to handle connections
2022-03-21 20:44:04,878 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-03-21 20:44:04,878 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-03-21 20:44:04,878 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
NOTICE: PHP message: PHP Warning:  Undefined variable $domains in /var/www/viewer/dmarcts-report-viewer.php on line 315
NOTICE: PHP message: PHP Warning:  Undefined variable $orgs in /var/www/viewer/dmarcts-report-viewer.php on line 316
NOTICE: PHP message: PHP Warning:  Undefined variable $periods in /var/www/viewer/dmarcts-report-viewer.php on line 317
NOTICE: PHP message: PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/viewer/dmarcts-report-viewer.php:125
Stack trace:
#0 /var/www/viewer/dmarcts-report-viewer.php(317): html()
#1 {main}
  thrown in /var/www/viewer/dmarcts-report-viewer.php on line 125
2022/03/21 20:44:18 [error] 24#24: *1 FastCGI sent in stderr: "PHP message: PHP Warning:  Undefined variable $domains in /var/www/viewer/dmarcts-report-viewer.php on line 315PHP message: PHP Warning:  Undefined variable $orgs in /var/www/viewer/dmarcts-report-viewer.php on line 316PHP message: PHP Warning:  Undefined variable $periods in /var/www/viewer/dmarcts-report-viewer.php on line 317PHP message: PHP Fatal error:  Uncaught TypeError: count(): Argument #1 ($var) must be of type Countable|array, null given in /var/www/viewer/dmarcts-report-viewer.php:125
Stack trace:
#0 /var/www/viewer/dmarcts-report-viewer.php(317): html()
#1 {main}
  thrown in /var/www/viewer/dmarcts-report-viewer.php on line 125" while reading response header from upstream, client: 172.18.0.19, server: _, request: "GET / HTTP/1.0", upstream: "fastcgi://127.0.0.1:9000", host: "dmarc.goose.ws"
172.18.0.19 - - [21/Mar/2022:20:44:18 -0400] "GET / HTTP/1.0" 200 720 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36" "-" 0.007 0.004 . -
[goose@Vergil: ~ ] $

The mailbox itself seems to be in fine working order:

[goose@Vergil: ~ ] $ docker exec -it dmarc-report /usr/bin/dmarcts-report-parser.pl -i -d -r
use tls without verify servercert.
connection to mail.[domain.tld] with Ssl => 0, User => dmarc@[domain.tld], Ignoresizeerrors => 1
Started at Mon Mar 21 20:44:11 2022
Using Mail::IMAPClient version 3.43 on perl 5.032000
Connecting with IO::Socket::IP PeerAddr mail.[domain.tld] PeerPort 143 Proto tcp Timeout 600 Debug 1
Connected to mail.[domain.tld]
Read:   * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Dovecot ready.
Sending: 1 STARTTLS
Sent 12 bytes
Read:   1 OK Begin TLS negotiation now.
Connecting with IO::Socket::IP PeerAddr mail.[domain.tld] PeerPort 143 Proto tcp Timeout 600 Debug 1
Connected to mail.[domain.tld]
Read:   * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Dovecot ready.
Sending: 2 STARTTLS
Sent 12 bytes
Read:   2 OK Begin TLS negotiation now.
Sending: 3 LOGIN dmarc@[domain.tld] [Redact: Count=3 Showcredentials=OFF]
Sent 66 bytes
Read:   3 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE LITERAL+ NOTIFY METADATA SPECIAL-USE COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in
Sending: 4 STATUS Inbox (MESSAGES)
Sent 27 bytes
Read:   * STATUS Inbox (MESSAGES 0)
        4 OK Status completed (0.002 + 0.000 + 0.001 secs).
Processing 0 messages in folder <Inbox>.
Sending: 5 STATUS Inbox (MESSAGES)
Sent 27 bytes
Read:   * STATUS Inbox (MESSAGES 0)
        5 OK Status completed (0.001 + 0.000 secs).
Sending: 6 LOGOUT
Sent 10 bytes
Read:   * BYE Logging out
        6 OK Logout completed (0.001 + 0.000 secs).
dmarcts-report-parser.pl: Processed 0 emails.
[goose@Vergil: ~ ] $

I've tried all the fixes from the previous issues/pull request, to no avail. Does anyone see where I'm going wrong?

stefangweichinger commented 2 years ago

Same for me, unfortunately.

gutmensch commented 2 years ago

@stefangweichinger @goose-ws Just to rule out the obvious (which is still an issue of the viewer I think): Do you have any single DMARC report in the database already or not? The viewer only works correctly AFAIR after the first report has been received and parsed into the database by the cronjob, which runs every 15 min. DMARC reports are async and triggered by the involved MX'es only after you have send some mails (can be daily or weekly, depending on the policies and how the MX respects those options).

mysql> select count(*) from dmarc_report.report;
+----------+
| count(*) |
+----------+
|     1690 |
+----------+
1 row in set (0.00 sec)

stefangweichinger commented 2 years ago

@gutmensch thanks for that feedback. I forwarded one mail to that new imap mailbox, but it seems my docker container has issues connecting via IMAP (using TLS, yes). Might be some docker or network issue here, I have to check that. (the container runs on the mailserver, and connects to IMAPs via a DNS record pointing to a WAN IP which is port-forwarded ... could be a problem).

stefangweichinger commented 2 years ago

@gutmensch solved here. Switched to using SSL, now the IMAP connection works, the first report was read and is displayed in the WebGUI of your application. Great, thanks!

goose-ws commented 2 years ago

@stefangweichinger @goose-ws Just to rule out the obvious (which is still an issue of the viewer I think): Do you have any single DMARC report in the database already or not? The viewer only works correctly AFAIR after the first report has been received and parsed into the database by the cronjob, which runs every 15 min. DMARC reports are async and triggered by the involved MX'es only after you have send some mails (can be daily or weekly, depending on the policies and how the MX respects those options).
mysql> select count(*) from dmarc_report.report;
+----------+
| count(*) |
+----------+
|     1690 |
+----------+
1 row in set (0.00 sec)

I had a single one, at the time I posted this issue. I've received a few more, while waiting for replies, and it appears to be working as expected now. Not sure if it was that, or something else. But I'll take it. Thanks.

gutmensch / docker-dmarc-report

Undefined variables #16