search

domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
990 stars 213 forks source link

ERROR:cli.py:606:IMAP Error: 'NoneType' object has no attribute 'decode' #156

Open MatthK opened 4 years ago

MatthK commented 4 years ago

Since a few days my system stopped updating and when I checked, the parsedmarc service is running into an error.

When running journalctl -u parsedmarc.service -r then I get a whole bunch of these error messages:

Mar 26 15:31:57 grafana systemd[1]: parsedmarc.service: Failed with result 'exit-code'.
Mar 26 15:31:57 grafana systemd[1]: parsedmarc.service: Main process exited, code=exited, status=1/FAILURE
Mar 26 15:31:57 grafana parsedmarc[23547]:    ERROR:cli.py:628:Connection timed out
Mar 26 15:29:46 grafana parsedmarc[23547]:    ERROR:cli.py:114:Splunk HEC error: HTTPSConnectionPool(host='splunkhec.example.com', port=443): Max r
Mar 26 15:29:46 grafana parsedmarc[23547]: [38B blob data]
Mar 26 15:29:29 grafana systemd[1]: Started parsedmarc mailbox watcher.
Mar 26 15:29:29 grafana systemd[1]: Stopped parsedmarc mailbox watcher.
Mar 26 15:29:29 grafana systemd[1]: parsedmarc.service: Scheduled restart job, restart counter is at 2210.
Mar 26 15:29:29 grafana systemd[1]: parsedmarc.service: Service hold-off time over, scheduling restart.

What is this error and how can I get it to work again?

j0sh-C commented 4 years ago

I think it is in this function, not validating tree in the except ?

(added the if statement in front to make mine keep parsing in a pinch..might just be imap)

def parse_aggregate_report_xml(xml, offline=False, nameservers=None, timeout=2.0, parallel=False): """Parses a DMARC XML report string and returns a consistent OrderedDict

Args:
    xml (str): A string of DMARC aggregate report XML
    offline (bool): Do not query online for geolocation or DNS
    nameservers (list): A list of one or more nameservers to use
    (Cloudflare's public DNS resolvers by default)
    timeout (float): Sets the DNS timeout in seconds
    parallel (bool): Parallel processing

Returns:
    OrderedDict: The parsed aggregate DMARC report
"""
errors = []
# Parse XML and recover from errors
try:
    xmltodict.parse(xml)["feedback"]
except Exception as e:
    errors.append("Invalid XML: {0}".format(e.__str__()))
    tree = etree.parse(BytesIO(xml.encode('utf-8')),
                       etree.XMLParser(recover=True))
    logger.debug('preetree')
    **if etree.tostring(tree):
        xml = etree.tostring(tree).decode('utf-8', errors='ignore')
    logger.debug('postetree')**
Thomas2500 commented 4 years ago

I do have the same problem when parsedmarc tried to read a MTA-STS mail. After deleting the MTA-STS mail, the program worked again.

Newly installed Ubuntu 18.04 with parsedmarc 6.9.0 via pip3:

$ parsedmarc --debug -c /etc/parsedmarc.ini
0it [00:00, ?it/s]
   DEBUG:__init__.py:1068:Found 2 messages in INBOX
   DEBUG:__init__.py:1072:Processing message 1 of 2: UID 1
   DEBUG:__init__.py:1072:Processing message 2 of 2: UID 2
   ERROR:cli.py:605:IMAP Error: 'NoneType' object has no attribute 'decode'
dragoangel commented 4 years ago

Want to note that on 6.7.4 I doesn't have such approach. All TLSA and MTA-STS with them are correctly moving to Invalid. I of course want them in Aggregate folder =D, but at least nothing is crashing.

pauloriply commented 3 years ago

Have the same error. After change email address in /etc/parsedmarc.ini FROM dmarc-reports@domain.com TO dmarcrecords@domain.com error is gone and service is UP. So i just delete "-" from my mailbox name.

davidem commented 3 years ago

I've had a similar error, caused by a "-" in the username/password for the elasticsearch endpoint. New user/password without "-" fixed it.