search

jgosmann / dmarc-metrics-exporter

Export Prometheus metrics from DMARC reports.
MIT License
42 stars 6 forks source link

Polling may be stuck #27

Closed sebPomme closed 1 year ago

sebPomme commented 1 year ago

Hi,

I can't manage do to the polling work.
I feel like It poll my mailbox only at startup.

Yesterday I have to manually restart the service to treat my dmarc reports.

To debug I set --debug on my command line. And today, I receive a new dmarc mail, nothing move on the mailbox. Nothing in the logs except GET on /metrics.

How can I debug more deeply ?

btw my "poll_interval_seconds": 60

Pomme

jgosmann commented 1 year ago

Fix released as version 0.6.1.

sebPomme commented 1 year ago

Thanks I will keep you in touch

sebPomme commented 1 year ago

Still not working, my conf

{
    "listen_addr": "0.0.0.0",
    "port": 9797,
    "imap": {
        "host": "exemple.imap-server.net",
        "port": 111,
        "username": "postmaster@blablabla.com",
        "password": "lol",
        "use_ssl": true,
        "verify_certificate": true
    },
    "folders": {
        "inbox": "INBOX",
        "done": "Archive",
        "error": "Invalid"
    },
    "storage_path": "/var/lib/dmarc-metrics-exporter",
    "poll_interval_seconds": 60,
    "deduplication_max_seconds": 604800
}

poll_interval_seconds set to 60s. 

dmarc_1     | 2022-07-19T13:34:11.089635376Z INFO:uvicorn.access:10.68.0.18:39932 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:35:11.090594465Z INFO:uvicorn.access:10.68.0.18:40080 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:36:11.086343746Z INFO:uvicorn.access:10.68.0.18:40426 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:37:11.091414440Z INFO:uvicorn.access:10.68.0.18:40518 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:38:11.090858613Z INFO:uvicorn.access:10.68.0.18:40678 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:39:11.090270833Z INFO:uvicorn.access:10.68.0.18:41104 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:40:11.090241996Z INFO:uvicorn.access:10.68.0.18:41258 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:41:11.085814898Z INFO:uvicorn.access:10.68.0.18:41478 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:42:11.089827571Z INFO:uvicorn.access:10.68.0.18:41950 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:43:11.090084405Z INFO:uvicorn.access:10.68.0.18:42038 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:44:11.089981781Z INFO:uvicorn.access:10.68.0.18:42144 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:45:11.090829683Z INFO:uvicorn.access:10.68.0.18:42496 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:46:11.085812999Z INFO:uvicorn.access:10.68.0.18:42682 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:47:11.089866608Z INFO:uvicorn.access:10.68.0.18:42734 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:48:11.090295566Z INFO:uvicorn.access:10.68.0.18:43190 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:49:11.089622079Z INFO:uvicorn.access:10.68.0.18:43296 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:50:11.090640123Z INFO:uvicorn.access:10.68.0.18:43406 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:51:11.088016775Z INFO:uvicorn.access:10.68.0.18:44176 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:52:11.089591903Z INFO:uvicorn.access:10.68.0.18:44286 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:53:11.090210565Z INFO:uvicorn.access:10.68.0.18:44350 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:54:11.089795087Z INFO:uvicorn.access:10.68.0.18:44674 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:55:11.090272685Z INFO:uvicorn.access:10.68.0.18:44780 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:56:11.087871831Z INFO:uvicorn.access:10.68.0.18:44958 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:57:11.089801804Z INFO:uvicorn.access:10.68.0.18:45372 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:58:11.094418141Z INFO:uvicorn.access:10.68.0.18:45620 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T13:59:11.092281717Z INFO:uvicorn.access:10.68.0.18:45724 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:00:11.090178268Z INFO:uvicorn.access:10.68.0.18:46132 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:01:11.087665006Z INFO:uvicorn.access:10.68.0.18:46302 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:02:11.087963524Z INFO:uvicorn.access:10.68.0.18:46342 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:03:11.090659567Z INFO:uvicorn.access:10.68.0.18:46724 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:04:11.090142224Z INFO:uvicorn.access:10.68.0.18:46982 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:05:11.089607182Z INFO:uvicorn.access:10.68.0.18:47020 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:06:11.085685554Z INFO:uvicorn.access:10.68.0.18:47394 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:07:11.090400291Z INFO:uvicorn.access:10.68.0.18:47470 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:08:11.089775595Z INFO:uvicorn.access:10.68.0.18:47574 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:09:11.089257843Z INFO:uvicorn.access:10.68.0.18:47970 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:10:11.091749428Z INFO:uvicorn.access:10.68.0.18:48078 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:11:11.086448082Z INFO:uvicorn.access:10.68.0.18:48252 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:12:11.089906696Z INFO:uvicorn.access:10.68.0.18:48626 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:13:11.090376355Z INFO:uvicorn.access:10.68.0.18:48740 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:14:11.089873390Z INFO:uvicorn.access:10.68.0.18:48844 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:15:11.089946060Z INFO:uvicorn.access:10.68.0.18:49180 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:16:11.085816576Z INFO:uvicorn.access:10.68.0.18:49364 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:17:11.089907855Z INFO:uvicorn.access:10.68.0.18:49628 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:18:11.089728250Z INFO:uvicorn.access:10.68.0.18:49748 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:19:11.090697858Z INFO:uvicorn.access:10.68.0.18:49814 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:20:11.086598159Z INFO:uvicorn.access:10.68.0.18:49954 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:21:11.086981466Z INFO:uvicorn.access:10.68.0.18:50596 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:22:11.089547607Z INFO:uvicorn.access:10.68.0.18:50634 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:23:11.090655475Z INFO:uvicorn.access:10.68.0.18:50750 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:24:11.090393847Z INFO:uvicorn.access:10.68.0.18:51158 - "GET /metrics HTTP/1.1" 200
dmarc_1     | 2022-07-19T14:25:11.089856439Z INFO:uvicorn.access:10.68.0.18:51262 - "GET /metrics HTTP/1.1" 200

My docker-compose file

  dmarc:
    image: jgosmann/dmarc-metrics-exporter:0.6.1
    restart: unless-stopped
    user: 1000:1000
    volumes:
      - '/lol/dmarc-metrics-exporter/dmarc-metrics-exporter.json:/etc/dmarc-metrics-exporte
r.json'
      - '/lol/dmarc-metrics-exporter:/var/lib/dmarc-metrics-exporter:rw'
    networks:
      - nope
    command:
      - "--debug"

And nothing more, the new dmarc mail from google is in my INBOX

Pomme

sebPomme commented 1 year ago

Unfortunately, I din't have the rights to re-open the issue

sebPomme commented 1 year ago

Fun story, I set a crontab to restart the exporter every 30min. And now I can see in the log the imap part works and poll the mailbox every 60 sec. I din't change anything since my last message, but ok. So we can say everything is ok now. :heavy_check_mark:

Thank you for the fix in v0.6.1

jgosmann commented 1 year ago

Great to hear!