[Bug]: Cannot install v3 on Ubuntu LTS properly

[BachateroJ]

Hello, im currently trying to install v3 on Ubuntu LTS, getting the following errors during installation: pip install prowler-cloud /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead   from cryptography.utils import int_from_bytes /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead   from cryptography.utils import int_from_bytes ERROR: Could not find a version that satisfies the requirement prowler-cloud (from versions: none) ERROR: No matching distribution found for prowler-cloudi

also tried to use the PyPi guide but it doesnt work properly

[toniblyx]

Hello @BachateroJ, thanks for let us know. What exact version of ubuntu LTS do you have and what python version is installed? Remember that Prowler 3 only works on python v3.9 and newer

[BachateroJ]

Hey, thanks for your response, i have 3.9 installed

python3.9 --version Python 3.9.16

NAME="Ubuntu" VERSION="20.04.3 LTS (Focal Fossa)"

[toniblyx]

Cool, thanks, we are gonna try to reproduce it first.

[toniblyx]

can you confirm what version of pip are you using? pip3 --version I'm trying to reproduce it with the same version as you and I get this one pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8) despite I have Python 3.9 installed.

[BachateroJ]

hey, i get the same output as you do

pip3 --version pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

[toniblyx]

Awesome, now we know what is going on. I solved locally. This is what it is happening, you need to make sure you are using pip for 3.9 (apparently by default Ubuntu takes pip 3.8).

1. Install python 3.9 sudo apt-get install python3.9 (you have this done)
2. Remove python 3.8 to avoid conflicts if you can sudo apt-get remove python3.8 (this is not mandatory)
3. Make sure you have the python3 distutils package installed sudo apt-get install python3-distutils
4. To make sure you use pip for 3.9 get the get-pip script curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
5. Execute it with the proper python version sudo python3.9 get-pip.py
6. Now you should have pip for 3.9 ready pip3.9 --version
7. Go ahead and install Prowler with pip3.9 install prowler-cloud
8. Test it with prowler -v you should get Prowler 3.0.1

Let me know if that works.

[BachateroJ]

thanks! it now seems that the installation ran smoothly, but once i use the prowler command, i get this error:

prowler -v prowler: command not found

[toniblyx]

Ok, if you run pip3.9 show -f prowler-cloud you will see all Prowler files and where they are located. Executable file prowler is in /usr/local/bin/prowler (you can also figure that out with whereis prowler). You system may not have /usr/local/bin in the $PATH. If you run echo $PATH you probably won't see /usr/local/bin if that is the case, that is the problem you are facing. To add it to your path do export PATH=$PATH:/usr/local/bin and for persistence echo "PATH=\$PATH:/usr/local/bin" >> ~/.bashrc

[BachateroJ]

hey, thank you again, this is the output of the echo: echo $PATH /sbin:/bin:/usr/bin:/usr/local/bin:/snap/bin:/usr/local/bin

these are the errors that i get prowler -v prowler: command not found

prowler aws prowler: command not found

[toniblyx]

where you able to see where are all files located with pip3.9 show -f prowler-cloud? you should see where prowler is and then why you can't call it. I have done the installation as I mentioned above and worked fine. if you have docker you can run prowler using docker too. If the pip install went well, it has to be something related to your path.

[BachateroJ]

this is the output i got: i cut the rest of it as it not relevant

pip3.9 show -f prowler-cloud Name: prowler-cloud Version: 3.0.1 Summary: Prowler is an Open Source security tool to perform Cloud Security best practices assessments, audits, incident response, continuous monitoring, hardening and forensics readiness. It contains more than 240 controls covering CIS, PCI-DSS, ISO27001, GDPR, HIPAA, FFIEC, SOC2, AWS FTR, ENS and custom security frameworks. Home-page: Author: Author-email: Toni de la Fuente toni@blyx.com License: Apache-2.0 Location: /home/assaf/.local/lib/python3.9/site-packages Requires: alive-progress, arnparse, azure-identity, azure-mgmt-authorization, azure-mgmt-security, azure-mgmt-storage, azure-mgmt-subscription, azure-storage-blob, boto3, botocore, colorama, detect-secrets, msgraph-core, pydantic, shodan, tabulate Required-by: Files: ../../../bin/prowler prowler/init.py prowler/main.py prowler/pycache/init.cpython-39.pyc prowler/pycache/main.cpython-39.pyc prowler/compliance/aws/init.py prowler/compliance/aws/pycache/init.cpython-39.pyc prowler/compliance/aws/cis_1.4_aws.json prowler/compliance/aws/cis_1.5_aws.json prowler/compliance/aws/ens_rd2022_aws.json prowler/compliance/azure/init.py prowler/compliance/azure/pycache/init.cpython-39.pyc prowler/config/init.py prowler/config/pycache/init.cpython-39.pyc prowler/config/pycache/config.cpython-39.pyc prowler/config/allowlist.yaml prowler/config/checklist_example.json prowler/config/config.py prowler/config/config.yaml prowler/lib/init.py prowler/lib/pycache/init.cpython-39.pyc prowler/lib/pycache/banner.cpython-39.pyc prowler/lib/pycache/logger.cpython-39.pyc prowler/lib/banner.py prowler/lib/check/init.py

[toniblyx]

Ok, you have it there, ../../../bin/prowler Run find / -type f -name prowler and you will see where exactly it is. It should be in /usr/local/bin/prowler but not sure in your system due to the errors you are getting.

[BachateroJ]

the find command made this output, is it fine?

sudo find / -type f -name prowler /home/assaf/.local/bin/prowler

[toniblyx]

That is totally fine since I guess you are running pip3.9 install as your user assaf so you have to execute /home/assaf/.local/bin/prowler to run Prowler or add /home/assaf/.local/bin/ to your path first and then run prowler.

Remember: export PATH=$PATH:/home/assaf/.local/bin/ and for persistence echo "PATH=\$PATH:/home/assaf/.local/bin/" >> ~/.bashrc then prowler

[BachateroJ]

done the following and tried prowler -v, this is the result:

prowler -v Traceback (most recent call last): File "/home/assaf/.local/bin/prowler", line 5, in from prowler.main import prowler File "/home/assaf/.local/lib/python3.9/site-packages/prowler/main.py", line 7, in from prowler.lib.check.check import ( File "/home/assaf/.local/lib/python3.9/site-packages/prowler/lib/check/check.py", line 15, in from prowler.lib.outputs.outputs import report File "/home/assaf/.local/lib/python3.9/site-packages/prowler/lib/outputs/outputs.py", line 15, in from prowler.lib.outputs.file_descriptors import fill_file_descriptors File "/home/assaf/.local/lib/python3.9/site-packages/prowler/lib/outputs/file_descriptors.py", line 22, in from prowler.providers.azure.lib.audit_info.models import Azure_Audit_Info File "/home/assaf/.local/lib/python3.9/site-packages/prowler/providers/azure/lib/audit_info/models.py", line 3, in from azure.identity import DefaultAzureCredential File "/usr/lib/python3/dist-packages/azure/identity/init.py", line 8, in from ._credentials import ( File "/usr/lib/python3/dist-packages/azure/identity/_credentials/init.py", line 8, in from .client_credential import CertificateCredential, ClientSecretCredential File "/usr/lib/python3/dist-packages/azure/identity/_credentials/client_credential.py", line 6, in from .._base import ClientSecretCredentialBase, CertificateCredentialBase File "/usr/lib/python3/dist-packages/azure/identity/_base.py", line 8, in from cryptography import x509 File "/usr/lib/python3/dist-packages/cryptography/x509/init.py", line 8, in from cryptography.x509.base import ( File "/usr/lib/python3/dist-packages/cryptography/x509/base.py", line 18, in from cryptography.x509.extensions import Extension, ExtensionType File "/usr/lib/python3/dist-packages/cryptography/x509/extensions.py", line 20, in from cryptography.hazmat.primitives import constant_time, serialization File "/usr/lib/python3/dist-packages/cryptography/hazmat/primitives/constant_time.py", line 11, in from cryptography.hazmat.bindings._constant_time import lib ModuleNotFoundError: No module named '_cffi_backend'

[toniblyx]

Your python installation on that machine seems to be incomplete or broken, I have found this in the internet https://stackoverflow.com/questions/34370962/no-module-named-cffi-backend. Apparently you need to install cfii using pip.

[BachateroJ]

that's odd because the system reports that the req is already satisfied

python3.9 -m pip install cffi Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: cffi in /usr/lib/python3/dist-packages (1.14.0)

[BachateroJ]

i forced a re-installation of cffi so this error is now gone but replaced with another: prowler -v Traceback (most recent call last): File "/home/assaf/.local/bin/prowler", line 5, in from prowler.main import prowler File "/home/assaf/.local/lib/python3.9/site-packages/prowler/main.py", line 38, in from prowler.providers.common.audit_info import set_provider_audit_info File "/home/assaf/.local/lib/python3.9/site-packages/prowler/providers/common/audit_info.py", line 16, in from prowler.providers.azure.azure_provider import Azure_Provider File "/home/assaf/.local/lib/python3.9/site-packages/prowler/providers/azure/azure_provider.py", line 5, in from azure.mgmt.subscription import SubscriptionClient File "/usr/lib/python3/dist-packages/azure/mgmt/subscription/init.py", line 12, in from ._configuration import SubscriptionClientConfiguration File "/usr/lib/python3/dist-packages/azure/mgmt/subscription/_configuration.py", line 11, in from msrestazure import AzureConfiguration File "/usr/lib/python3/dist-packages/msrestazure/init.py", line 28, in from .azure_configuration import AzureConfiguration File "/usr/lib/python3/dist-packages/msrestazure/azure_configuration.py", line 34, in from msrest import Configuration File "/home/assaf/.local/lib/python3.9/site-packages/msrest/init.py", line 28, in from .configuration import Configuration File "/home/assaf/.local/lib/python3.9/site-packages/msrest/configuration.py", line 38, in from .universal_http.requests import ( File "/home/assaf/.local/lib/python3.9/site-packages/msrest/universal_http/init.py", line 53, in from ..exceptions import ClientRequestError, raise_with_traceback File "/home/assaf/.local/lib/python3.9/site-packages/msrest/exceptions.py", line 31, in from azure.core.exceptions import SerializationError, DeserializationError ImportError: cannot import name 'SerializationError' from 'azure.core.exceptions' (/usr/lib/python3/dist-packages/azure/core/exceptions.py)

am i missing something? are there any dependencies that might have been not installed properly?

[toniblyx]

For some reason that I don't know your installation is not properly satisfying dependencies. It is so weird, we have tested it in many different OS and we didn't see your errors.

We will add the information below to the official Prowler documentation site https://docs.prowler.cloud/. As you can see there are many ways and OS where you can use Prowler, try something different. The quickest and simplest way may be using Docker or AWS CloudShell:

Installation

Prowler is available as a project in PyPI, thus can be installed using pip with Python >= 3.9:

=== "Generic"

_Requirements_:

* `Python >= 3.9`
* `Python pip >= 3.9`

_Commands_:

``` bash
pip install prowler-cloud
prowler -v
```

=== "Docker"

_Requirements_:

* Have `docker` installed: https://docs.docker.com/get-docker/.
* In the command below, change `-v` to your local directory path in order to access the reports.

_Commands_:

``` bash
docker run -ti --rm -v /your/local/dir/prowler-output:/home/prowler/output \
--name prowler \
--env AWS_ACCESS_KEY_ID \
--env AWS_SECRET_ACCESS_KEY \
--env AWS_SESSION_TOKEN toniblyx/prowler:latest
```

=== "Ubuntu 20.04.3 LTS"

_Requirements_:

* Install python 3.9 with: `sudo apt-get install python3.9`
* Remove python 3.8 to avoid conflicts if you can: `sudo apt-get remove python3.8`
* Make sure you have the python3 distutils package installed: `sudo apt-get install python3-distutils`
* To make sure you use pip for 3.9 get the get-pip script with: `curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py`
* Execute it with the proper python version: `sudo python3.9 get-pip.py`
* Now you should have pip for 3.9 ready: `pip3.9 --version`

_Commands_:

```
pip3.9 install prowler-cloud
export PATH=$PATH:/home/$HOME/.local/bin/
prowler -v
```

=== "Amazon Linux 2"

_Requirements_:

* Latest Amazon Linux 2 should come with Python 3.9 already installed however it may need pip. Install Python pip 3.9 with: `sudo dnf install -y python3-pip`.
* Make sure setuptools for python is already installed with: `pip3 install setuptools`

_Commands_:

```
pip3.9 install prowler-cloud
export PATH=$PATH:/home/$HOME/.local/bin/
prowler -v
```

=== "AWS CloudShell"

Prowler can be easely executed in AWS CloudShell but it has some prerequsites to be able to to so. AWS CloudShell is a container running with `Amazon Linux release 2 (Karoo)` that comes with Python 3.7, since Prowler requires Python >= 3.9 we need to first install a newer version of Python. Follow the steps below to successfully execute Prowler v3 in AWS CloudShell:

- First install all dependences and then Python, in this case we need to compile it because there is not a package available at the time this document is written:
```
sudo yum -y install gcc openssl-devel bzip2-devel libffi-devel
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz
tar zxf Python-3.9.16.tgz
cd Python-3.9.16/
./configure --enable-optimizations
sudo make altinstall
python3.9 --version
cd 
```
- Once Python 3.9 is available we can install Prowler from pip:
```
pip3.9 install prowler-cloud
```
- Now enjoy Prowler:
```
prowler -v
prowler 
```

- To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/home/cloudshell-user/output/prowler-output-123456789012-20221220191331.csv`

_Requirements_:

* `Python >= 3.9`
* `Python pip >= 3.9`

_Commands_:

``` bash
pip install prowler-cloud
prowler -v
```

[BachateroJ]

is there a way to run a check which dependencies are missing and install them?

[toniblyx]

do you mean a prowler check? the errors you are getting are due to dependencies that for some reason are not getting installed in your environment.

You can also try to execute prowler like python3.9 /home/assaf/.local/bin/prowler to make sure python 3.9 is used

[BachateroJ]

i managed to fix that: prowler -v Prowler 3.0.1

here is what i did: sudo apt autoremove - i thought that some dependencies previously installed caused a conflict or interrupted the installation sudo apt-get install -f - installing all dependencies that i might have missed

tried prowler -v and got the following error: ModuleNotFoundError: No module named 'isodate'

fixed with pip install isodate

after installing isodate it started working properly, running a test scan on an azure environment

ive been using this tool for a long time on my aws environment and now that the new version also supports azure it gets me pumped up, waiting for the GCP update :)

thank you for your help!

[toniblyx]

Woohooo!! Great news! Thanks for let us know what you did and glad to see that it is working now for you.

Yeah, 2023 is gonna be amazing for Prowler and the community! ;)

[toniblyx]

leaving this comment here to make sure @BachateroJ reads it, we have now a branch with the first integration with GCP including a bunch of checks, it is here https://github.com/prowler-cloud/prowler/pull/2125 please feel free to test it if you like.

[BachateroJ]

thank you very much! will look into it! we are deploying a new GCP environment in about a month or so, so i will use it once we finish the deployment