Splunk Python Scripts - Githubissues

1an2132145 commented 3 years ago

So executing the following, as per Splunk’s instructions:

Chmod +x deploy_attack_range.sh ./deploy_attack_range.sh [This script is successful, no errors found so dependencies are happy]

Then trying to build it: Python attack_range_local.py –a build [This errors out with the below] Traceback (most recent call last): File “attack_range_local.py”, line 4, in From modules import logger ImportError: No module named modules

So I tried Python3 and this time stops on an import in line 7: Python3 attack_range_local.py –a build [Error below] Traceback (Most recent call last): File “attack_range_local.py”, line 7, in From modules.VagrantController import VagrantController File “/home/DEVICE/attackrange/attack_range_local/modules/VagrantController.py”, Line 2, in From jinja2 import Environment, FileSystemLoader ModuleNotFoundError: No Module named ‘jinja2’

So no matter using Python 2 or 3, it’s not happy with the Py scripts provided by Splunk. We can’t remove the offending module imports because this would have a knock on effect on the code in the body of the script, as it’s written using those modules.

All support welcome.

P4T12ICK commented 3 years ago

Hello, this is super weird. First of all, the result of python ... or python3 ... shouldn't be different because you specified python3 in your virtaulenv.

Can you check multiple things:

Is your virtualenv activated when you run the command?
In which folder are you, when you run the command?
Can you perform the steps manually from the bash script? (especially check if jinja2 was installed using pip)

1an2132145 commented 3 years ago

Hello - thanks for the update, we can confirm:

Yes the virtualenv is activated;
we’re in the root attack range folder where the deploy script resides; &
yes we’ve tried manual (the script runs fine with no errors though so that would suggest manual is also fine).

P4T12ICK commented 3 years ago

This is weird. When pip installed the packages properly, you should have the jinja2 module.

P4T12ICK commented 3 years ago

Did you check if jinja2 was under the installed packages?

1an2132145 commented 3 years ago

hello yes it is.

and the versioning: python-jinja2/bionic-updates,bionic-updates,bionic-security,bionic-security,now 2.10-1ubuntu0.18.04.1 all [installed]

(venv) Machine@name:~/usernameattackrange/attack_range_local$ pip install -r requirements.txt Requirement already satisfied: ansible==2.9.2 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (2.9.2) Requirement already satisfied: ansible-runner==1.4.4 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 2)) (1.4.4) Requirement already satisfied: Jinja2==2.10.3 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 3)) (2.10.3) Requirement already satisfied: python-vagrant==0.5.15 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 4)) (0.5.15) Requirement already satisfied: pywinrm==0.4.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 5)) (0.4.1) Requirement already satisfied: PyYAML==5.3.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (5.3.1) Requirement already satisfied: splunk-sdk==1.6.12 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 7)) (1.6.12) Requirement already satisfied: tabulate==0.8.6 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 8)) (0.8.6) Requirement already satisfied: cryptography in ./venv/lib/python3.6/site-packages (from ansible==2.9.2->-r requirements.txt (line 1)) (3.3.1) Requirement already satisfied: PyYAML==5.3.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (5.3.1) Requirement already satisfied: Jinja2==2.10.3 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 3)) (2.10.3) Requirement already satisfied: python-daemon in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (2.2.4) Requirement already satisfied: pexpect>=4.5 in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (4.8.0) Requirement already satisfied: PyYAML==5.3.1 in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (5.3.1) Requirement already satisfied: six in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (1.15.0) Requirement already satisfied: psutil in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (5.8.0) Requirement already satisfied: MarkupSafe>=0.23 in ./venv/lib/python3.6/site-packages (from Jinja2==2.10.3->-r requirements.txt (line 3)) (1.1.1) Requirement already satisfied: requests-ntlm>=0.3.0 in ./venv/lib/python3.6/site-packages (from pywinrm==0.4.1->-r requirements.txt (line 5)) (1.1.0) Requirement already satisfied: six in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (1.15.0) Requirement already satisfied: xmltodict in ./venv/lib/python3.6/site-packages (from pywinrm==0.4.1->-r requirements.txt (line 5)) (0.12.0) Requirement already satisfied: requests>=2.9.1 in ./venv/lib/python3.6/site-packages (from pywinrm==0.4.1->-r requirements.txt (line 5)) (2.25.1) Requirement already satisfied: cffi>=1.12 in ./venv/lib/python3.6/site-packages (from cryptography->ansible==2.9.2->-r requirements.txt (line 1)) (1.14.4) Requirement already satisfied: six in ./venv/lib/python3.6/site-packages (from ansible-runner==1.4.4->-r requirements.txt (line 2)) (1.15.0) Requirement already satisfied: pycparser in ./venv/lib/python3.6/site-packages (from cffi>=1.12->cryptography->ansible==2.9.2->-r requirements.txt (line 1)) (2.20) Requirement already satisfied: ptyprocess>=0.5 in ./venv/lib/python3.6/site-packages (from pexpect>=4.5->ansible-runner==1.4.4->-r requirements.txt (line 2)) (0.7.0) Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (from python-daemon->ansible-runner==1.4.4->-r requirements.txt (line 2)) (51.1.1) Requirement already satisfied: docutils in ./venv/lib/python3.6/site-packages (from python-daemon->ansible-runner==1.4.4->-r requirements.txt (line 2)) (0.16) Requirement already satisfied: lockfile>=0.10 in ./venv/lib/python3.6/site-packages (from python-daemon->ansible-runner==1.4.4->-r requirements.txt (line 2)) (0.12.2) Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm==0.4.1->-r requirements.txt (line 5)) (2020.12.5) Requirement already satisfied: chardet<5,>=3.0.2 in ./venv/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm==0.4.1->-r requirements.txt (line 5)) (4.0.0) Requirement already satisfied: idna<3,>=2.5 in ./venv/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm==0.4.1->-r requirements.txt (line 5)) (2.10) Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv/lib/python3.6/site-packages (from requests>=2.9.1->pywinrm==0.4.1->-r requirements.txt (line 5)) (1.26.2) Requirement already satisfied: ntlm-auth>=1.0.2 in ./venv/lib/python3.6/site-packages (from requests-ntlm>=0.3.0->pywinrm==0.4.1->-r requirements.txt (line 5)) (1.5.0) Requirement already satisfied: cryptography in ./venv/lib/python3.6/site-packages (from ansible==2.9.2->-r requirements.txt (line 1)) (3.3.1) Requirement already satisfied: requests>=2.9.1 in ./venv/lib/python3.6/site-packages (from pywinrm==0.4.1->-r requirements.txt (line 5)) (2.25.1)

Karma1331 commented 3 years ago

Hi @1an2132145 just to be sure because I had the same issue when trying to build before activating venv. Can you try the following commands in that order please?

cd attack_range_local
virtualenv -p python3 venv
source venv/bin/activate
python attack_range_local.py -a build

tdrlocal commented 3 years ago

Probably bit out of topic, but is there any major differences if building in Ubuntu 20.04 compared to supposedly 18.04?

FLRD0 commented 3 years ago

Hello, I had the same issue but manged to solve it doing a manual installation; follow the steps in the installation script file and you'll be fine. Also make sure you are in the right folder in your terminal and the virtualenv venv is also set for your python environment and then build the environment - on your terminal you sould see something like this: (venv) yourusername@machinename[...]

1an2132145 commented 3 years ago

Hi - thanks for the tips. I'll go through this again. RE: the username in the example above was redacted.

julianwieg commented 3 years ago

So executing the following, as per Splunk’s instructions:

Chmod +x deploy_attack_range.sh ./deploy_attack_range.sh [This script is successful, no errors found so dependencies are happy]

Then trying to build it: Python attack_range_local.py –a build [This errors out with the below] Traceback (most recent call last): File “attack_range_local.py”, line 4, in From modules import logger ImportError: No module named modules

So I tried Python3 and this time stops on an import in line 7: Python3 attack_range_local.py –a build [Error below] Traceback (Most recent call last): File “attack_range_local.py”, line 7, in From modules.VagrantController import VagrantController File “/home/DEVICE/attackrange/attack_range_local/modules/VagrantController.py”, Line 2, in From jinja2 import Environment, FileSystemLoader ModuleNotFoundError: No Module named ‘jinja2’

So no matter using Python 2 or 3, it’s not happy with the Py scripts provided by Splunk. We can’t remove the offending module imports because this would have a knock on effect on the code in the body of the script, as it’s written using those modules.

All support welcome.

I went straight to the reply from Karma1331 as I forgot to load the virtualenv and then it worked

1an2132145 commented 3 years ago

All - thank you for your comments.

We have reset the VM and increased its sizing. Current errors: Attack_range_error1 Attack_range_error2

kseidenschnur commented 3 years ago

Are running on a nonGUI OS?

1an2132145 commented 3 years ago

Hi - we've spoken to Splunk and we will close this case down and go down the AWS path. Thank you all for support and comments.

splunk / attack_range_local

Splunk Python Scripts #31