jorritfolmer
commented
1 year ago Thanks for reporting this issue. Unfortunately I can't help you with the IMAP error without additional context. E.g. which email provider? How many messages are in the INBOX? Did it ever work? What did the logs say then? What was the add-on logging in previous lines when it failed?
Regarding the UID lookup: it shouldn't grow exponentially. It should only grow when a new message is seen and downloaded. If it does grow exponentially then I assume you will also be seeing lots of duplicate events? If that is the case it means the UIDs the IMAP returns are not persistent across connections. I never seen that before. Can you share more information about the IMAP server?
Splunk Version - 8.1.3 App Version 3.2.4
Also, |inputlookup ta_dmarc_checkpointer_lookup is growing exponentially. can I suggest you to have latest entry in checkpoint instead of having all entries of UIDs that are read already.
2022-08-31 10:24:18,978 ERROR pid=8253 tid=MainThread file=base_modinput.py:log_error:309 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/lib/python3.7/imaplib.py", line 979, in _command self.send(data + CRLF) File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/tls.py", line 60, in send self.sock.sendall(data) File "/opt/splunk/lib/python3.7/ssl.py", line 1034, in sendall v = self.send(byte_view[count:]) File "/opt/splunk/lib/python3.7/ssl.py", line 1003, in send return self._sslobj.write(data) BrokenPipeError: [Errno 32] Broken pipe During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-dmarc/bin/ta_dmarc/aob_py3/modinput_wrapper/base_modinput.py", line 128, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc_imap.py", line 92, in collect_events input_module.collect_events(self, ew) File "/opt/splunk/etc/apps/TA-dmarc/bin/input_module_dmarc_imap.py", line 50, in collect_events filelist = i2d.process_incoming() File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 304, in process_incoming response = self.get_dmarc_message_bodies(new_messages) File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 135, in get_dmarc_message_bodies set(messageslist[x:x + fetch_size]), [b'RFC822'])) File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/imapclient.py", line 1329, in fetch tag = self._imap._command(*args) File "/opt/splunk/lib/python3.7/imaplib.py", line 981, in _command raise self.abort('socket error: %s' % val) imaplib.IMAP4.abort: socket error: [Errno 32] Broken pipe