search

BC-SECURITY / Empire

Empire is a post-exploitation and adversary emulation framework that is used to aid Red Teams and Penetration Testers.
https://bc-security.gitbook.io/empire-wiki/
BSD 3-Clause "New" or "Revised" License
4.21k stars 575 forks source link

nameserver: discover nameserver within environment of this host #741

Closed cmitcho closed 4 weeks ago

cmitcho commented 1 month ago

Describe your changes

Add a nameserver check for Linux hosts.

Issue ticket number and link (if there is one)

Checklist before requesting a review

cmitcho commented 1 month ago

Please let me know if documentation is needed or unit tests. If you would like tests, let me know where to read and I'll jump on that. Thanks!

BuildAndDestroy commented 1 month ago

Please let me know if documentation is needed or unit tests. If you would like tests, let me know where to read and I'll jump on that. Thanks!

I figured this out: https://github.com/BC-SECURITY/Empire/blob/main/.github/CONTRIBUTING.md#code-formatting-and-linting

BuildAndDestroy commented 1 month ago

Evidence of linting:

ubuntu@ubuntu2204-empire:~/development/git/Empire$ git branch 
  main
* nameserver
ubuntu@ubuntu2204-empire:~/development/git/Empire$ poetry run ruff check . --fix 
All checks passed!
ubuntu@ubuntu2204-empire:~/development/git/Empire$ poetry run black .
All done! ✨ 🍰 ✨
390 files left unchanged.

There seems to be a problem with pytest. When running a pytest, including on older versions, I am getting errors. I also ran the docker-compose.yml tests from the .github workflow, they fail as well. Also note, docker-compose no longer exists, it is now docker compose.

BuildAndDestroy commented 1 month ago

Snippet of errors from pytest, they seem to be the same in docker compose and poetry:

ubuntu@ubuntu2204-empire:~/development/git/Empire$ sudo poetry run pytest
[sudo] password for ubuntu: 
==================================================== test session starts =====================================================
platform linux -- Python 3.12.2, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/ubuntu/development/git/Empire
configfile: pytest.ini
plugins: timeout-2.3.1, cov-4.1.0, anyio-4.4.0
collected 347 items                                                                                                          

empire/test/test_agent_api.py EEEEEEEE                                                                                 [  2%]
empire/test/test_agent_checkins_api.py sEsss                                                                           [  3%]
empire/test/test_agent_file_api.py EEEEEEE                                                                             [  5%]
empire/test/test_agent_task_api.py EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE                           [ 22%]
empire/test/test_agent_task_service.py E                                                                               [ 22%]
empire/test/test_agents.py EEEEEEEEEEEEEEEEEEEE                                                                        [ 28%]
empire/test/test_bypass_api.py EEEEEEEEEEE                                                                             [ 31%]
empire/test/test_common_agents.py E                                                                                    [ 31%]
empire/test/test_config.py E                                                                                           [ 31%]
empire/test/test_credential_api.py EEEEEEEEEE                                                                          [ 34%]
empire/test/test_download_api.py EEEEEEE                                                                               [ 36%]
empire/test/test_download_service.py EE                                                                                [ 37%]
empire/test/test_helpers.py E                                                                                          [ 37%]
empire/test/test_hooks.py EEEEE                                                                                        [ 39%]
empire/test/test_hooks_internal.py E                                                                                   [ 39%]
empire/test/test_host_api.py EEE                                                                                       [ 40%]
empire/test/test_host_process_api.py EEEEE                                                                             [ 41%]
empire/test/test_listener_api.py EEEEEEEEEEEEEEEEEEEE                                                                  [ 47%]
empire/test/test_listener_generate_launcher.py EEEEEEEE                                                                [ 49%]
empire/test/test_logs.py EEEEEE                                                                                        [ 51%]
empire/test/test_meta_api.py E                                                                                         [ 51%]
empire/test/test_module_api.py EEEEEEEEEEE                                                                             [ 55%]
empire/test/test_module_service.py EEEEE                                                                               [ 56%]
empire/test/test_modules.py sEEE                                                                                       [ 57%]
empire/test/test_obfuscation_api.py EEEEEEEEEEEEEEEEsEE                                                                [ 63%]
empire/test/test_openapi.py E                                                                                          [ 63%]
empire/test/test_option_util.py EEEEEEEEEEEEEEEEEEE                                                                    [ 68%]
empire/test/test_plugin_api.py EEEEEEEEEEEEEEE                                                                         [ 73%]
empire/test/test_plugin_service.py EEEEE                                                                               [ 74%]
empire/test/test_plugin_task_api.py EEEEE                                                                              [ 76%]
empire/test/test_profile_api.py EEEEEEEEE                                                                              [ 78%]
empire/test/test_socket_hooks.py EE                                                                                    [ 79%]
empire/test/test_stager_api.py EEEEEEEEEEEEEEEEEEEEEEEEEEE                                                             [ 87%]
empire/test/test_startup_loaders.py EEEE                                                                               [ 88%]
empire/test/test_string_util.py EE                                                                                     [ 88%]
empire/test/test_tags_api.py EEEEEEEEEEEEEEEEEEEEEE                                                                    [ 95%]
empire/test/test_user_api.py EEEEEEEEEEEEEEE                                                                           [ 99%]
empire/test/test_zz_reset.py ss                                                                                        [100%]

=========================================================== ERRORS ===========================================================
_________________________________________ ERROR at setup of test_get_agent_not_found _________________________________________

client = <starlette.testclient.TestClient object at 0x7fd55be50bf0>
admin_auth_header = {'Authorization': 'Bearer <REDACTED>}
..
..
..
vinnybod commented 1 month ago

@BuildAndDestroy The important part for debugging the pytest errors is below the summary, if you want to share that I can see if it looks familiar.

Regardless, we don't currently require testing specific modules. The code looks fine to me, so if the CI passes we can merge it.

cmitcho commented 1 month ago

As soon as the bug was fixed, pytest completes without issue:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================== 339 passed, 8 skipped, 750 warnings in 214.18s (0:03:34) ==============================================

We should be good to run the workflow

cmitcho commented 1 month ago

Looks like we skip Pytest coverage comment in Test Python 3.10 but we run in Test Python 3.12. Is there a manual way to run this locally? I assumed this would have been under sudo poetry run pytest but the output shows no reference.

Also commented this on the other pull request, we are hitting the same in PR 742.

cmitcho commented 1 month ago

I was able to track this down in the Workflow file:

DATABASE_USE=mysql poetry run pytest -v --runslow --cov=empire/server --junitxml=pytest.xml --cov-report=term-missing:skip-covered . | tee pytest-coverage.txt

Ran this locally with sudo, most of ther Failures went away except for 1:

68 files skipped due to complete coverage.

=========================== short test summary info ============================
FAILED empire/test/test_stager_api.py::test_pyinstaller_stager_creation - Per...
=========== 1 failed, 346 passed, 762 warnings in 934.21s (0:15:34) ============

Looking through the log output:

=================================== FAILURES ===================================
_______________________ test_pyinstaller_stager_creation _______________________

client = <starlette.testclient.TestClient object at 0x73e67f4c6b70>
pyinstaller_stager = {'name': 'MyStager3', 'options': {'Language': 'python', 'Listener': 'new-listener-1', 'OutFile': 'empire', 'SafeChecks': 'True', ...}, 'template': 'multi_pyinstaller'}
admin_auth_header = {'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJlbXBpcmVhZG1pbiIsImV4cCI6MTcyNDUyNzk5Nn0.B4Bj8G0BGCemB04-ZegULS9UI6vwi9LfV4xqZfdUmvw'}

    def test_pyinstaller_stager_creation(client, pyinstaller_stager, admin_auth_header):
>       response = client.post(
            "/api/v2/stagers/?save=true", headers=admin_auth_header, json=pyinstaller_stager
        )

empire/test/test_stager_api.py:437: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
..
..
>       with open(binary_file_str + ".py", "w") as text_file:
E       PermissionError: [Errno 13] Permission denied: '/tmp/empire.py'

empire/server/stagers/multi/pyinstaller.py:153: PermissionError

Permission error. Odd since I am running sudo and this is in /tmp/ and the file permissions are my lower level user:

ubuntu@ubuntu-Parallels-Virtual-Platform:~/development/git/Empire$ ls -alh /tmp/empire.py 
-rw-rw-r-- 1 ubuntu ubuntu 1.6K Aug 23 11:23 /tmp/empire.py

So I went back and modified perms:

ubuntu@ubuntu-Parallels-Virtual-Platform:~/development/git/Empire$ sudo chown -R ubuntu:ubuntu *

ubuntu@ubuntu-Parallels-Virtual-Platform:~/development/git/Empire$ sudo chown -R ubuntu:ubuntu .pytest_cache/

ubuntu@ubuntu-Parallels-Virtual-Platform:~/development/git/Empire$ sudo chown -R ubuntu:ubuntu .coverage

Looks like this resolved the Failures:

-----------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                           17671   9329    47%

68 files skipped due to complete coverage.

================ 347 passed, 765 warnings in 865.63s (0:14:25) =================

As for the poetry test, we are reading the same in the workflow as we are locally, so this may not be the problem. We can see the next test is sqlite. Let's run it:

ubuntu@ubuntu-Parallels-Virtual-Platform:~/development/git/Empire$ DATABASE_USE=sqlite poetry run pytest . -v --runslow
..
..
============================================== 346 passed, 1 skipped, 760 warnings in 605.13s (0:10:05) ==============================================

No Errors heres. Seems to be a problem with the Workflow. If a resource is not accessible by the integration, is it really an error or a warning that can be skipped?