search

mitre-atlas / arsenal

CALDERA plugin for adversary emulation of AI-enabled systems
https://mitre-atlas.github.io/arsenal/
Apache License 2.0
85 stars 8 forks source link

venv dependency not found when running via caldera-atlas #1

Open lilyjw opened 1 year ago

lilyjw commented 1 year ago

Describe the bug Arsenal encounters an error about not having python3.10-venv available during start within caldera-atlas usage, i.e. CALDERA in a Docker container.

Not sure where to address - is there a way to define additional dependencies for CALDERA to install from a plugin? Or is this something caldera-atlas should do.

To Reproduce Steps to reproduce the behavior:

Set up caldera-atlas

  1. git clone --recursive https://github.com/mitre-atlas/caldera-atlas If one has SSH keys uploaded, this should check out all submodules.
    Otherwise, there may be failures because some submodules have SSH URLs. To resolve: a. git checkout https to get the branch with HTTPS URLs b. git submodule sync --recursive && git submodule update --init

  2. docker-compose build If one gets an OpenSSL UnsafeLegacyRenegotiation error, resolve it as one sees fit but can add this to caldera-atlas/caldera/Dockerfile ~L18 after the RUN apt-get ...

    # OpenSSL 3.0 disables UnsafeLegacyRenegotiation by default, must re-enable it for some endpoints (see https://github.com/dotnet/runtime/issues/80641)
RUN sed -i 's/providers = provider_sect/providers = provider_sect\n\
    ssl_conf = ssl_sect\n\
    \n\
    [ssl_sect]\n\
    system_default = system_default_sect\n\
    \n\
    [system_default_sect]\n\
    Options = UnsafeLegacyRenegotiation/' /etc/ssl/openssl.cnf

  3. docker-compose up -d

The container exits shortly after this - see the trace below

Expected behavior Container stays up

Trace From the Docker container log of CALDERA (the caldera-atlas service):

2023-03-15 16:37:41 - INFO  (app_svc.py:116 load) Enabled plugin: access
2023-03-15 16:37:41 - DEBUG (arsenal_svc.py:122 _create_venv)
Building cf_venv...
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

2023-03-15 16:37:41 - ERROR (base_events.py:1744 default_exception_handler) Task exception was never retrieved
future: <Task finished name='Task-33' coro=<AppService.load_plugins.<locals>.load() done, defined at /usr/src/app/app/service/app_svc.py:108> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/usr/lib/python3.10/venv/__init__.py", line 318, in _setup_pip
    subprocess.check_output(
  File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1896, in _run_once
    handle._run()
  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/app/app/service/app_svc.py", line 115, in load
    await plugin.enable(self.get_services())
  File "/usr/src/app/app/objects/c_plugin.py", line 67, in enable
    await plugin(services)
  File "/usr/src/app/plugins/arsenal/hook.py", line 15, in enable
    await arsenal_svc.populate_venv(env_name='cf_venv', with_counterfit=True)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 62, in populate_venv
    await self._create_venv(env_name=env_name)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 136, in _create_venv
    builder.create(env_dir=env_dir)
  File "/usr/lib/python3.10/venv/__init__.py", line 75, in create
    self._setup_pip(context)
  File "/usr/lib/python3.10/venv/__init__.py", line 336, in _setup_pip
    sys.exit(1)
SystemExit: 1

Desktop (please complete the following information): Ubuntu 22.04 but running CALDERA as a container using its own Dockerfile, which uses ubuntu:latest

Additional context Add any other context about the problem here.

mdotter-mitre commented 1 year ago

@lilyjw Core plugins to caldera have dependencies installed via a requirements file during the build of the docker image - as this is a stand-alone plugin at this time, we need to build the Dockerfile with these dependencies added.

Thank you for identifying this bug. I'll create a PR to update the repo https://github.com/mitre-atlas/caldera-atlas with python3.10-venv included!

lilyjw commented 1 year ago

Temporary workaround: Add python3-venv to the RUN apt-get ... step in the caldera/Dockerfile, then re-run docker-compose build