splunk / eventgen

Splunk Event Generator: Eventgen
Apache License 2.0
380 stars 179 forks source link

[FEATURE/IMPROVEMENT] Enabling jinja template with splunk eventgen app (eventgen as splunk app) #357

Open siddharthajuprod07 opened 4 years ago

siddharthajuprod07 commented 4 years ago

Is your feature request related to a problem? Please describe. Yes. Splunk eventgen app is not able to initialize jinja template because the code is resolving to a path "$SPLUNK_HOME\etc\apps\SA-Eventgen\lib\plugins\generator" , which doesn't exists. Below is my environment details, OS : Windows Splunk Version : 8.0.1 Eventgen Version : 6.5.2 (as eventgen 7 still not work with splunk 8) python version : both python2 and python3 having same issue.

Describe the solution you'd like Splunkbase eventgen should work with jinja template.

Describe alternatives you've considered I did some findings from my end to fix this but no luck. I followed the below steps after seeing the code of eventgen , roughly in the below order modinput_eventgen.py >> eventgen_core.py >> eventgentimer.py >> eventgenconfig.py >> eventgenexceptions.py >> eventgen_core.py (as the exception PluginNotLoaded handled here) && jinja.py

The steps I followed,

  1. Under SA-Eventgen\lib I created the folder structure plugins\generator (as the code is looking for this path).
  2. Under generator folder I copied the jinja2 folder from SA-Eventgen\lib folder as jinja.py is looking for that.
  3. I also copied jinja.py from $SPLUNK_HOME\etc\apps\SA-Eventgen\lib\plugins\generator folder to SA-Eventgen\lib\plugins\generator folder.
  4. I also created an empty init.py file in SA-Eventgen\lib\plugins\generator folder.
  5. Restarted splunk.

Now I am receiving the below error.

02-06-2020 18:02:32.343 +0530 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\Python2.exe" "C:\Program Files\Splunk\etc\apps\SA-Eventgen\bin\modinput_eventgen.py"" 2020-02-06 18:02:31 eventgen ERROR MainProcess {'exception': 'Traceback (most recent call last):\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\splunk_eventgen\eventgen_core.py", line 336, in _initializePlugins\n module = imp.load_module(base, mod_name, mod_path, mod_desc)\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\plugins\generator\jinja.py", line 8, in \n from jinja2 import nodes\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\init.py", line 33, in \n from jinja2.environment import Environment, Template\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\environment.py", line 15, in \n from jinja2 import nodes\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\nodes.py", line 19, in \n from jinja2.utils import Markup\n File "C:\Program Files\Splunk\etc\apps\SA-Eventgen\lib\jinja2\utils.py", line 647, in \n from markupsafe import Markup, escape, soft_unicode\nImportError: No module named markupsafe', 'event': ImportError('No module named markupsafe',)}

The error is coming from utils.py under jinja2 in last line,

from markupsafe import Markup, escape, soft_unicode

Initially I was thinking this is related to python issue but when I changed the python version in server.conf its not able to initialize the modular input.

Additional context None.

li-wu commented 4 years ago

@siddharthajuprod07 could you try the latest version of Eventgen 7.1.0 on Splunk 8.0 on Linux platform that solves this issue? Currently there is a bug in Splunk preventing it working properly on Windows platform.