| Author | Oluwaseun Remi-Omosowon |
|---|---|
| App Version | 1.6.0 |
| Vendor Products |
|
The TA-mailclient add-on fetches emails for Splunk to index from mailboxes using either POP3 or IMAP, with or without SSL.
The modular input also stores takes the password from inputs.conf in plain text, and replaces it with a place holder, while storing it encrypted within Splunk. This is built using the Splunk SDK for Python, should work on any Splunk installation with Python available including SHC. Passwords should also get replicated between search heard peer members.
This only fetches emails from the 'inbox' folder when using POP3. Additional mailbox folders can be indexed when using IMAP.
Be sure to set the interval to run this as frequently as required.
It supports all 'text/*' content types and several well known scripts (.bat, .js, .sh) detailed below:
'application/xml'
'application/xhtml'
'application/x-sh'
'application/x-csh',
'application/javascript'
'application/bat'
'application/x-bat'
'application/x-msdos-program'
'application/textedit'Images, videos and executables are not indexed.
Includes:
Version 1.6.0 of the TA-mailclient is compatible with:
| Splunk Enterprise versions | 8.x, 7.x |
|---|---|
| CIM | Not Applicable |
| Platforms | Platform independent |
| Lookup file changes | No lookups included in this app |
This version removes support for unencrypted connections to mailboxes to allow the app pass Splunk Certification. The _issecure is no longer required and should be removed from the config.
The administrator is responsible for setting the sourcetype to whatever is desired, as well as extracting CIM fields for the sourcetype. This app already includes several extractions for different parts of the message that can be reused.
This app will not work on a universal forwarder, as it requires Python which comes with an HF or a full Splunk install.
Note: Travis CI includes tests for both secure versions of POP3 / IMAP.
TA-mailclient includes the following new features:
This is currently tested against 7.3, 8.0 and the latest version of Splunk Enterprise (v8.1 as at the time of this writing). Issues can be reported and tracked on Github at this time.
This uses the inbuilt poplib and imaplib that comes with Python by default.
Contributions on github are welcome and will be incorporated into the main release. Current contributors are listed in AUTHORS.md.
v1.6.0
v1.5.5
v1.4.0
v1.3.5
v1.3.0
v0.5.1
v0.5.0
v0.4.9
v0.4.8
v0.4.7
v0.4.6
v0.4.5
v0.4.4
v0.4.3
Mail.save_password() to allow reuse of code when writing other modular inputs.v0.4.2
v0.4.1
v0.4
Note: filename and filecontent are multi-valve fields.
v0.3
v0.2
Questions and answers
Access questions and answers specific to the TA-mailclient at (https://answers.splunk.com/).
Support
This Splunk support add-on is community / developer supported.
Questions asked on Splunk answers will be answered either by the community of users or by the developer when available. All support questions should include the version of Splunk and OS.
You can also contact the developer directly via Splunkbase. Feedback and feature requests can also be sent via Splunkbase.
Issues can also be submitted at the TA-mailclient repo via on Github
Future release will support
Note : This has not been tested against an exhaustive list of mail servers, so I'll welcome the feedback.
Also, feel free to send me a list of well known servers that you 're using this with without problems.
Rate the add-on on Splunkbase if you use it and are happy with it, and share your feedback. Thanks!
TA-mailclient supports the following server platforms in the versions supported by Splunk Enterprise:
The app was developed to be platform agnostic, but tests are mostly run on Linix.
Please contact the developer with issues running this on Windows. See the Splunk documentation for hardware requirements for running a heavy forwarder.
To function properly, TA-mailclient has no external requirements but needs to be installed on a full Splunk install which provides python and the required libraries (poplib and imaplib).
Because this add-on runs on Splunk Enterprise, all of the Splunk Enterprise system requirements apply.
Download the TA-mailclient at one of the following locaitons:
To install and configure this app on your supported standalone platform, do one of the following:
$SPLUNK_HOME/etc/apps/ and restart SplunkInstall to search head - (Standalone or Search head cluster)
Install to indexers
Install to forwarders
Follow the steps to install the TA-mailclient on a heavy forwarder. More instructions available at the following URL
Configure an email input by going to the setup page or configuring inputs.conf.
For Splunk cloud installations, install TA-mailclient on a heavy forwarder that has been configured to forward events to your Splunk Cloud instance. The sourcetype is set by the administrator of the heavy forwarder when configuring the inputs.
You can work with Splunk Support on installing the Support add-on on Splunk Cloud for parsing the mails collected.
This app adds a mail:// modular input and supports a variety of parameters in inputs.conf.
[mail://email_address@domain.com]
interval = 600
mailserver = imap.domain.com
password = mypassword
protocol = IMAP|POP3
disabled = 0
mailbox_cleanup = delete
additional_folder = test,rfc,spam
Once the input is read, the password gets replaced and shows as 'encrypted'. As such, the password for the mailbox must not be set to 'encrypted'.
The input can be edited if the password needs to be updated, and the password stored in a password storage endpoint would get updated automatically. Passwords are never stored in clear text.
A different sourcetype can be specified for each input, thus making it possible to have different sourcetypes for every mailbox. Mailbox cleanup is also managed automatically, and emails are deleted once it has been indexed.
mailserver - This is a mandatory field and should be the hostname or IP address for the mail server or client access server with support for retrieving emails via POP3 or IMAP
protocol - This must be set to either POP3 or IMAP
password - Passwords must be set for every account, or the input will get disabled.
mailbox_cleanup = This indicates if every email should be deleted as it is read,
or delayed until the next interval.
Setting this to readonly prevents mails from being deleted.
The default is readonly. Supported options are:
delayed|delete|readonly
interval - This should be configured to run as frequent as required to retreive emails. This modular input retrieves up to 20 emails at each run. A future release to this input might allow the limit to be configured as a parameter to the modular input.
This modular input supports multiple instances, and each input runs at separate intervals.
include_headers - This determines if email headers should be included.
additional_folders - This is an optional parameter containing a comma-separated list of additional folders to be indexed if IMAP is configured for the mailbox.
drop_attachment - This is an optional parameter to determine if email attachment should be discarded.
A copy of the Creative Commons Legal code has been added to the add-on detailing its license.
Data is indexed using a sourcetype specified by the administrator when configuring the inputs.
If nothing is specified, events will get indexed with a sourcetype of mail.
Once an email is indexed, it will not be re-indexed except the checkpoint directory is emptied. This can be achieved by running the following command:
splunk clean inputdata mailLogs can be found by searching Splunk internal logs
index=_internal sourcetype=splunkd (component=ModularInputs OR component=ExecProcessor) mail.py
Additional logging can be enabled by turning on debug logging for ExecProcessor and ModInputs. set the logging level of the ExecProcessor to Debug
/opt/splunk/bin/splunk set log-level ExecProcessor -level DEBUG /opt/splunk/bin/splunk set log-level ModInputs -level DEBUG
You can find additional ways to enable debug logging on here.