Unable to run Caldera 5.x on Ubuntu 20.04.6 LTS using either docker build or standard build

[robertstrom]

Describe the bug

I built a brand new virgin Ubuntu 20.04.6 VM that is fully updated and then walked through the instructions to install and run the latest version of Caldera.

When building the Docker instance I am unable to login using the documented credentials.

When attempting to launch Caldera non-Docker instance it fails to start

To Reproduce Steps to reproduce the behavior:

• Docker attempted build

  
  curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
  sudo apt-get install -y nodejs
  sudo apt install python3-pip

git clone https://github.com/mitre/caldera.git --recursive cd caldera/ sudo apt install docker.io sudo usermod -aG docker $USER newgrp docker docker build . --build-arg WIN_BUILD=true -t caldera:latest

at this point I am able to get to a Caldera logon web page but none of the credentials are working for logon (credentials from the default.yaml file)

users: blue: blue: admin red: admin: admin red: admin

- Install / run non-Docker version

NOTE that dependencies are already installed as shown above.

- Launching with the following command

python3 server.py --insecure --build

![image](https://github.com/mitre/caldera/assets/3240205/1ce70105-1869-46ec-9913-f0431ad07140)

![image](https://github.com/mitre/caldera/assets/3240205/dfdc4fc7-6d46-41ca-b592-3545fa7e124f)

Error text

Traceback (most recent call last): File "server.py", line 281, in run_tasks(services=app_svc.get_services(), run_vue_server=args.uiDevHost) File "server.py", line 77, in run_tasks loop.run_until_complete(app_svc.register_contacts()) File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete return future.result() File "/home/rstrom/caldera/app/service/app_svc.py", line 150, in register_contacts await self.register_contact_tunnels(contact_svc) File "/home/rstrom/caldera/app/service/app_svc.py", line 155, in register_contact_tunnels tunnel_class = import_module(tunnel_module_name).Tunnel File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 848, in exec_module File "", line 219, in _call_with_frames_removed File "/home/rstrom/caldera/app/contacts/tunnels/tunnel_ssh.py", line 3, in import asyncssh File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/init.py", line 31, in from .agent import SSHAgentClient, SSHAgentKeyPair, connect_agent File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/agent.py", line 34, in from .public_key import KeyPairListArg, SSHCertificate, SSHKeyPair File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/public_key.py", line 35, in from .crypto import ed25519_available, ed448_available File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/crypto/init.py", line 58, in from .x509 import X509Certificate, X509Name, X509NamePattern File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/crypto/x509.py", line 32, in from OpenSSL import crypto File "/usr/lib/python3/dist-packages/OpenSSL/init.py", line 8, in from OpenSSL import crypto, SSL File "/usr/lib/python3/dist-packages/OpenSSL/crypto.py", line 1553, in class X509StoreFlags(object): File "/usr/lib/python3/dist-packages/OpenSSL/crypto.py", line 1571, in X509StoreFlags NOTIFY_POLICY = _lib.X509_V_FLAG_NOTIFY_POLICY AttributeError: module 'lib' has no attribute 'X509_V_FLAG_NOTIFY_POLICY'

**Expected behavior**

- For the Docker build - Be able to log into Caldera
- For the standard build / execution - have Caldera start

**Screenshots**

Screenshots in-line above

**Desktop (please complete the following information):**
 - OS - Ubuntu server

NAME="Ubuntu" VERSION="20.04.6 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.6 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal

 - Browser - Chrome - Version 122.0.6261.129 (Official Build) (64-bit)

**Additional context**
- When running the Docker instance I am connecting to Caldera from a remote machine using the URL https://ipaddress:8888
- I also read another issues thread that updated npm using these commands and the build / execution still failed. Running the commands below put the npm version at **v20.11.1**

sudo npm cache clean -f sudo npm install -g n sudo n stable

[github-actions[bot]]

Looks like your first issue -- we aim to respond to issues as quickly as possible. In the meantime, check out our documentation here: http://caldera.readthedocs.io/

[robertstrom]

FYI - I built out another Ubuntu 20.04.6 LTS VM and recursively cloned the 4.2.0 branch and built the Docker container and have it running just fine. I am able to connect to it and login as I would expect to be able to do with version 5.

[TacitTactics]

I have tried to run on Ubuntu 23.10 the installation is successful and login appear but no activity when clicking the login button. I then tried to roll back to 4.2.0 the installation did not work at all for Ubuntu 22.04. It breaks at vite.js requiring a newer version of VUE. Manual installation of the VUE package does not affect the installation. I will try Ubuntu 20.04 with 4.2.0 and see what happens.

[TacitTactics]

I concur with @robertstrom, 4.2.0 on 20.04 works as expected. There is an incompatibility running 5.0.0 on any of the Ubuntu flavors, and an incompatibility between 4.2.0 and Ubuntu 22.04 +

[elegantmoose]

@TacitTactics Are you pulling 5.0.0 or Master. If the former, try pulling Master. We have putting patches in.

Also - this issue may apply here as well -> https://github.com/mitre/caldera/issues/2885

[TacitTactics]

Thank you for suggestion. I will try the master branch with both 22.04 and 23.10 and report back. To be clear, pulling from the master is leaving off the " --recursive" option, correct?

[elegantmoose]

Correct

[robertstrom]

cloned without the --recursive argument and I am getting this error

strom@caldera-5-ubuntu-2004:~/caldera$ python3 server.py --insecure --build
2024-03-25 20:29:58 WARNING  --insecure flag set. Caldera will use the default user accounts in default.yml config   server.py:216
                             file.                                                                                                
                    INFO     Using main config from conf/default.yml                                                 server.py:225
2024-03-25 20:29:59 INFO     Building VueJS front-end.                                                               server.py:261
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: eslint-plugin-jinja2@0.1.0
npm ERR! Found: eslint@8.50.0
npm ERR! node_modules/eslint
npm ERR!   dev eslint@"^8.50.0" from the root project
npm ERR!   peer eslint@"^6.0.0 || ^7.0.0 || >=8.0.0" from @eslint-community/eslint-utils@4.4.0
npm ERR!   node_modules/@eslint-community/eslint-utils
npm ERR!     @eslint-community/eslint-utils@"^4.2.0" from eslint@8.50.0
npm ERR!   2 more (eslint-config-airbnb-base, eslint-plugin-import)
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer eslint@"^0.23.0 || ^1.0.0 || ^2.0.0" from eslint-plugin-jinja2@0.1.0
npm ERR! node_modules/eslint-plugin-jinja2
npm ERR!   dev eslint-plugin-jinja2@"^0.1.0" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: eslint@2.13.1
npm ERR! node_modules/eslint
npm ERR!   peer eslint@"^0.23.0 || ^1.0.0 || ^2.0.0" from eslint-plugin-jinja2@0.1.0
npm ERR!   node_modules/eslint-plugin-jinja2
npm ERR!     dev eslint-plugin-jinja2@"^0.1.0" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /home/rstrom/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/rstrom/.npm/_logs/2024-03-25T20_29_59_540Z-debug-0.log
Traceback (most recent call last):
  File "server.py", line 262, in <module>
    subprocess.run(["npm", "install"], cwd="plugins/magma", check=True)
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['npm', 'install']' returned non-zero exit status 1.

[robertstrom]

If you do not pull the repository recursively you get the error below when you try to build the Docker container

rstrom@caldera-5-ubuntu-20:~/caldera$ docker build . --build-arg WIN_BUILD=true -t caldera:latest
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  7.662MB
Step 1/44 : FROM ubuntu:23.04
23.04: Pulling from library/ubuntu
6360b3717211: Pull complete 
Digest: sha256:5a828e28de105c3d7821c4442f0f5d1c52dc16acf4999d5f31a3bc0f03f06edd
Status: Downloaded newer image for ubuntu:23.04
 ---> f4cdeba72b99
Step 2/44 : SHELL ["/bin/bash", "-c"]
 ---> Running in cbc00e3fa120
Removing intermediate container cbc00e3fa120
 ---> e93864fd5709
Step 3/44 : ARG TZ="UTC"
 ---> Running in deda115b86ff
Removing intermediate container deda115b86ff
 ---> 0e70b34ce51a
Step 4/44 : RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime &&     echo $TZ > /etc/timezone
 ---> Running in cf9afe446031
Removing intermediate container cf9afe446031
 ---> fb06543e92be
Step 5/44 : WORKDIR /usr/src/app
 ---> Running in 41e7e8031afc
Removing intermediate container 41e7e8031afc
 ---> 78101b512c12
Step 6/44 : ADD . .
 ---> 9f321a0604b3
Step 7/44 : RUN if [ -z "$(ls plugins/stockpile)" ]; then echo "stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again."; exit 1; fi
 ---> Running in 10258785599f
stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again.
The command '/bin/bash -c if [ -z "$(ls plugins/stockpile)" ]; then echo "stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again."; exit 1; fi' returned a non-zero code: 1

[robertstrom]

I tried building the docker container after downloading the repository recursively. The build was successful but the same problem (not able to login) still exists

[Jaynornj]

I had the same issue with my 5.0.0

[elegantmoose]

@robertstrom You are pulling Master branch down recursively correct? (i.e. git clone https://github.com/mitre/caldera.git --recursive without specifying version. Do not specify v 5.0.0, you want Master as patches are being pushed there. When stable we will mint a version 5.1.0)

*FWIW, you cannot run Caldera without using the recursive flag, as some of the plugins have become a core dependency in v5, mainly the Magma plugin which is the entire new UI.

[elegantmoose]

@Jaynornj Same, try pulling Master, not version 5.0.0.

[rasmus1514]

@elegantmoose Also having the same problems even when pulling down the Master through "git clone https://github.com/mitre/caldera.git --recursive" does not seem to solve the issue sadly.

[dathtd119]

Ayo, I got this problem with V5.x on my Ubuntu 20.0.4 LTS. And you know what, I redo, but in server.py, i changed npm to pnpm, and boom, Solved!

I changed them at these

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[Jaynornj]

Use another version for a quick solution

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

[ssamiev]

I have also issue with version 5.0.0.

[ssamiev]

remote: Compressing objects: 100% (742/742), done.
error: RPC failed; curl 18 transfer closed with outstanding read data remaining error: 6407 bytes of body are still expected fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: fetch-pack: invalid index-pack output fatal: clone of 'https://github.com/mitre/sandcat.git' into submodule path '/home/admininfosec/caldera/plugins/sandcat' failed Failed to clone 'plugins/sandcat'. Retry scheduled

[ssamiev]

error: RPC failed; curl 18 transfer closed with outstanding read data remaining error: 7419 bytes of body are still expected fetch-pack: unexpected disconnect while reading sideband packet fatal: early EOF fatal: fetch-pack: invalid index-pack output fatal: clone of 'https://github.com/mitre/sandcat.git' into submodule path '/home/admininfosec/caldera/plugins/sandcat' failed Failed to clone 'plugins/sandcat' a second time, aborting

[Jaynornj]

try with another version!