search

rsmusllp / king-phisher

Phishing Campaign Toolkit
BSD 3-Clause "New" or "Revised" License
2.22k stars 538 forks source link

Hangs on INFO: Synchronizing King Phisher's pipenv environment #357

Closed BenU closed 5 years ago

BenU commented 5 years ago

Issue Description

I'm installing King Physher as part of TrustedSec's The PenTester's Framework: PTF Github page

It got hung up each time at the INFO: Synchronizing King Phisher's pipenv environment point despite having googled that error message and attempting to install python, python3, cairo, etc. Based on issue #353 I also tried: cd /opt/king-phisher sudo ./KingPhisherServer -L DEBUG -f ./server_config.yml Which produced the error: ModuleNotFoundError: No module named 'termcolor' More context below.

Reproduction Steps

sudo su - git clone https://github.com/trustedsec/ptf /opt/ptf cd /opt/ptf && ./ptf use modules/exploitation/install_update_all The above freezes at: INFO: Synchronizing King Phisher's pipenv environment

Environment Details

Host OS: Ubuntu 18.04 LTS King Phisher Version: x.y.z -- not sure of this. I assume the latest version.

Error Details / Stack Trace

Note that the trace below got hung up for over an hour at INFO: Synchronizing King Phisher's pipenv environment and was interrupted by me with ^C and exit which I also included because of additional data generated. 

...
adwaita-icon-theme-full is already the newest version (3.28.0-1ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
gir1.2-vte-2.91 is already the newest version (0.52.2-1ubuntu1~18.04.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
INFO: Successfully installed gir1.2-webkit2-4.0 with apt-get
INFO: Synchronizing Python package dependencies from PyPi
pip 19.0.3 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
Requirement already up-to-date: pip in /usr/local/lib/python3.6/dist-packages (19.0.3)
pip 19.0.3 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
Requirement already up-to-date: setuptools in /usr/local/lib/python3.6/dist-packages (40.8.0)
Requirement already up-to-date: six in /usr/local/lib/python3.6/dist-packages (1.12.0)
Requirement already satisfied: pipenv in /usr/local/lib/python3.6/dist-packages (2018.11.26)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.6/dist-packages (from pipenv) (16.4.3)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /usr/local/lib/python3.6/dist-packages (from pipenv) (0.5.1)
Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (40.8.0)
Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (19.0.3)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from pipenv) (2018.1.18)
INFO: Synchronizing King Phisher's pipenv environment
^Cexit
^CTraceback (most recent call last):
  File "./KingPhisher", line 45, in <module>
    sys.exit(main())
  File "./KingPhisher", line 42, in main
    return startup.pipenv_entry(parser, os.path.basename(__file__))
  File "/opt/king-phisher/king_phisher/startup.py", line 161, in pipenv_entry
    results = _run_pipenv(('--site-packages', 'install'), cwd=target_directory)
  File "/opt/king-phisher/king_phisher/startup.py", line 87, in _run_pipenv
    results = run_process(args, cwd=cwd)
  File "/opt/king-phisher/king_phisher/startup.py", line 203, in run_process
    process_handle.wait()
  File "/usr/lib/python3.6/subprocess.py", line 1457, in wait
    (pid, sts) = self._try_wait(0)
  File "/usr/lib/python3.6/subprocess.py", line 1404, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
INFO: Installing the client desktop application file
INFO: Updating the GTK icon cache
gtk-update-icon-cache: Cache file created successfully.
exit
INFO: Configuring the PostgreSQL server
INFO: PostgreSQL configuration file found at /etc/postgresql/10/main/pg_hba.conf
WARNING: The PostgreSQL king_phisher user already exists! The password will need
WARNING: to be manually updated to match the one automatically generated by this
WARNING: installation script (by using the ALTER ROLE PostgreSQL command) or
WARNING: restored from a previously backed up server configuration file.
INFO: Installing the King Phisher systemd service file in /lib/systemd/system/
INFO: Starting the King Phisher service
Job for king-phisher.service failed because the control process exited with error code.
See "systemctl status king-phisher.service" and "journalctl -xe" for details.
--------------------------------------------------------------------------------
INFO: You can start the King Phisher client with the following command:
/opt/king-phisher/KingPhisher

Trying the install as described in #353 generates the following:

ubuntu@ip-XXX:~$ sudo su -
root@ip-XXX:~# cd /opt/king-phisher
root@ip-XXX:/opt/king-phisher# sudo ./KingPhisherServer -L DEBUG -f ./server_config.yml
DEBUG    target diretory: /opt/king-phisher
INFO     checking for the pipenv environment
DEBUG    pipenv path: '/usr/local/bin/pipenv'
DEBUG    pipenv Pipfile: /opt/king-phisher/Pipfile
Loading .env environment variables…
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/king-phisher/king_phisher/server/__main__.py", line 42, in <module>
    from king_phisher import color
  File "/opt/king-phisher/king_phisher/color.py", line 41, in <module>
    import termcolor
ModuleNotFoundError: No module named 'termcolor'
root@ip-XXX:/opt/king-phisher# exit
logout

Very grateful for this project and everything the King Phisher team does. Thank you!

wolfthefallen commented 5 years ago

From the information you provided, it appears you did a keyboard interrupt while King Phisher was setting up its pipenv environment. Steps to have King Phisher reinstall its pipenv dependencies is located in the FAQ

zeroSteiner commented 5 years ago

The installation process could hang on the pipenv step for quite awhile, upwards of 15 minutes if I remember correctly. About how long did you wait at that phase?

You should be able to do the following, as root from in the /opt/king-phisher directory:

  1. First remove the existing pipenv environment, it's probably corrupted so start fresh
    • pipenv --rm
  2. Next reinstall the pipenv environment, this time you should see more output as it's working
    • pipenv install
  3. Finally, re-run the installation script which will just note that the pipenv environment is already setup
    • tools/install.sh

That last step should work as all of the OS packages have already been installed by the first run on the installer. @wolfthefallen do you know of any that are missing, given that this user is running the master branch and not dev?

BenU commented 5 years ago

Very grateful to you both. I couldn't figure out how to implement the commands described in the FAQ's in this environment though that might work.

I waited over an hour previously and, with your note, @zeroSteiner, I waited over 16 minutes again. Still, I did implement the commands you describe which got hung up as well for over 10 minutes at the point below:

ubuntu@ip-XXX:~$ sudo su -
root@ip-XXX:~# cd /opt/king-phisher/
root@ip-XXX:/opt/king-phisher# pipenv --rm
Removing virtualenv (/opt/king-phisher/.venv)…
root@ip-XXX:/opt/king-phisher# pipenv install
Creating a virtualenv for this project…
Pipfile: /opt/king-phisher/Pipfile
Using /usr/bin/python3 (3.6.7) to create virtualenv…
⠏ Creating virtual environment...Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /root/.local/share/virtualenvs/king-phisher-RbupqVtz/bin/python3
Also creating executable in /root/.local/share/virtualenvs/king-phisher-RbupqVtz/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment! 
Virtualenv location: /root/.local/share/virtualenvs/king-phisher-RbupqVtz
Installing dependencies from Pipfile.lock (49fa41)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 15/66 — 00:00:21

After around 10 minutes:

An error occurred while installing advancedhttpserver==2.1.0 --hash=sha256:2dd28a788aaaebcf7aa7a2bfb788e6bdbf02f85684e5c0ac27b588b15288e479! 
Will try again.
An error occurred while installing alembic==1.0.2 --hash=sha256:04bcb970ca8659c3607ddd8ffd86cc9d6a99661c9bc590955e8813c66bfa582b! 
Will try again.

This stuff mean anything?

wolfthefallen commented 5 years ago

If pipenv has issues installing a dependency, It will let you know and then attempt to install them again after all other pypi packages have completed, as @zeroSteiner stated it can take sometime to complete.

BenU commented 5 years ago

Time did seem to help the pipenv install though I had to restart it several times and give it hours. Still, tools/install.sh hangs at INFO: Synchronizing King Phisher's pipenv environment which I let run for 12 hours overnight. More context below. I guess no-one else is having this problem so perhaps the problem is within TrustedSec's Pen Testing Framework.. Thanks again for everything.

...gir1.2-vte-2.91 is already the newest version (0.52.2-1ubuntu1~18.04.2).
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
INFO: Successfully installed gir1.2-webkit2-4.0 with apt-get
INFO: Synchronizing Python package dependencies from PyPi
pip 19.0.3 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
Requirement already up-to-date: pip in /usr/local/lib/python3.6/dist-packages (19.0.3)
pip 19.0.3 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
Requirement already up-to-date: setuptools in /usr/local/lib/python3.6/dist-packages (40.8.0)
Requirement already up-to-date: six in /usr/local/lib/python3.6/dist-packages (1.12.0)
Requirement already satisfied: pipenv in /usr/local/lib/python3.6/dist-packages (2018.11.26)
Requirement already satisfied: virtualenv-clone>=0.2.5 in /usr/local/lib/python3.6/dist-packages (from pipenv) (0.5.1)
Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (40.8.0)
Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.6/dist-packages (from pipenv) (19.0.3)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.6/dist-packages (from pipenv) (16.4.3)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from pipenv) (2018.1.18)
INFO: Synchronizing King Phisher's pipenv environment
wolfthefallen commented 5 years ago

by chance are you running through a proxy?

BenU commented 5 years ago

Yes. On aws lightsail per the setup in The Hacker Playbook 3: Practical Guide To Penetration Testing. Is that the problem?

wolfthefallen commented 5 years ago

Ok I think I know what the issue is. pipenv is hanging when trying to connect with pypi because ssl certificate is failing through the proxy. Try the following to see if it will resolve the issue. do export PIP_TRUSTED_HOST=pypi.org then try reinstalling.

zeroSteiner commented 5 years ago

So are were we able to get this issue resolved or are we still trying to figure out why it's hanging. Strictly speaking, I don't think we've tested KP on AWS Lights and depending on the flavor of Linux it may not be supported.

BenU commented 5 years ago

Sorry for delay. I haven't yet tried that last step and you can mark this issue as closed. Lightsail on AWS offers Ubuntu LTS 16.04 and 18.04 and I was using 18.04.

I'm really very impressed by the great work you both do with King Phisher and how patient and responsive you've been to an insignificant user like me. I'm sure that the internet is much more secure because of your work. Thank you.

tl;dr I confess that I set the project aside, bothered by the book, The Hacker Playbook 3's, errors. I don't expect a technology book to be completely up-to-date and error free but it should have a dynamic errata page to correct issues and/or the community to discuss work-arounds like those needed here. I got stuck here within the first pages in the 1st chapter and can't be the only person having this problem. I suspect the author uses some VPN besides lightsail on AWS and wrote about light sail because there's a cheap intro level without testing it or maybe Ubuntu 16.04 without specifying which Ubuntu.

zeroSteiner commented 5 years ago

We all start somewhere and we're glad you decided to try out King Phisher. If you have more issues in the future just open another ticket and we'll help you out.