search

avilum / secimport

eBPF Python runtime sandbox with seccomp (Blocks RCE).
https://avilum.github.io/secimport/
MIT License
183 stars 12 forks source link

Usage on Debian based systems #10

Closed seanmcfeely closed 1 year ago

seanmcfeely commented 2 years ago

Hello,

I don't understand the following since the documentation show that the attribute exists: https://docs.python.org/3.9/library/importlib.html

>>> os = secure_import('os', allow_shells=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dev/secimport_testing/venv/lib/python3.9/site-packages/secimport/sandbox_helper.py", line 50, in secure_import
    assert run_dtrace_script_for_module(
  File "dev/secimport_testing/venv/lib/python3.9/site-packages/secimport/sandbox_helper.py", line 81, in run_dtrace_script_for_module
    module_file_path = create_dtrace_script_for_module(
  File "dev/secimport_testing/venv/lib/python3.9/site-packages/secimport/sandbox_helper.py", line 128, in create_dtrace_script_for_module
    module = importlib.machinery.PathFinder().find_spec(module_name)
AttributeError: module 'importlib' has no attribute 'machinery'

UPDATE: This is not the issue of concern, see comments.

seanmcfeely commented 2 years ago

FYI, with py3.10 I do not get the above error.

However, I get the following:

>>> from secimport import secure_import
>>> 
>>> 
>>> os = secure_import('os', allow_shells=False)
/usr/bin/dtrace invalid option -e
Usage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dev/secimport_testing/venv/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 50, in secure_import
    assert run_dtrace_script_for_module(
  File "/dev/secimport_testing/venv/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 81, in run_dtrace_script_for_module
    module_file_path = create_dtrace_script_for_module(
  File "/dev/secimport_testing/venv/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 170, in create_dtrace_script_for_module
    assert (
AssertionError: Failed to compile the dtrace script at /tmp/.secimport/sandbox_os.d

I fear DTrace is not stable enough on Linux distros (I'm on debian base) to use secimport; perhaps there is something that could change in the script to support this linux DTrace variant? I installed DTrace from here: https://packages.debian.org/stretch/systemtap-sdt-dev

Let me know if you have any thoughts. Thanks!

avilum commented 2 years ago

Hi @seanmcfeely, Thanks again for the issue!

For the first snippet - I have not been able to reproduce this error. Is there any chance you ran python (sometimes defaults to python 2.7) instead of python3? This attribute 'machinery' has been around for a while as you said, It is not something new.

As for the second snippet, It seems like the dtrace CLI installed in systemtap-sdt-dev is not equivalent to the MacOS/Solaris dtrace CLI.

I will add the logic to run the proper command on linux, out of the box. Meanwhile - can you please try running the following quick tutorial and see if dtrace works on your machine properly together with your python interpreter? https://docs.python.org/3/howto/instrumentation.html#enabling-the-static-markers

Thanks, Avi

seanmcfeely commented 2 years ago

Hello @avilum,

Thanks for the quick response and sorry for the delay. I'm back at the keyboard today after a long weekend.

I ran through that instrumentation by installing CPython --with-dtrace in a docker container. I grabbed this Dockerfile and updated it to install systemtap-sdt-dev and then configure python --with-dtrace.

After that, I spun up a container and dropped in. I did the following:

root@a64506c303b1:/# python3 -m pip install secimport
Collecting secimport
  Downloading secimport-0.4.3-py3-none-any.whl (20 kB)
Collecting PyYAML<7.0,>=6.0
  Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 kB 14.1 MB/s eta 0:00:00
Installing collected packages: PyYAML, secimport
Successfully installed PyYAML-6.0 secimport-0.4.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Next:

Python 3.10.7 (main, Sep  8 2022, 01:46:46) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from secimport import secure_import
>>> 
>>> 
>>> os = secure_import('os', allow_shells=False)
sh: 1: dtrace: not found
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 50, in secure_import
    assert run_dtrace_script_for_module(
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 81, in run_dtrace_script_for_module
    module_file_path = create_dtrace_script_for_module(
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 170, in create_dtrace_script_for_module
    assert (
AssertionError: Failed to compile the dtrace script at /tmp/.secimport/sandbox_os.d

UPDATE: I noticed dtrace was not installed because the same Dockerfile removes it later via an apt purge auto-remove. I installed it in the container and ran this again:

Python 3.10.7 (main, Sep  8 2022, 01:46:46) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from secimport import secure_import
>>> os = secure_import('os', allow_shells=False)
/usr/bin/dtrace invalid option -e
Usage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 50, in secure_import
    assert run_dtrace_script_for_module(
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 81, in run_dtrace_script_for_module
    module_file_path = create_dtrace_script_for_module(
  File "/usr/local/lib/python3.10/site-packages/secimport/sandbox_helper.py", line 170, in create_dtrace_script_for_module
    assert (
AssertionError: Failed to compile the dtrace script at /tmp/.secimport/sandbox_os.d

Next, I did make sure the SystemTap static markers are present (built with --enable-shared):

# apt-get install -y binutils 
...

# readelf -S /usr/local/lib/libpython3.10.so.1.0  | grep .note.stapsdt
  [29] .note.stapsdt     NOTE             0000000000000000  003ad720
seanmcfeely commented 2 years ago

I forgot to answer your first question; I was using Python 3.9.13 in a virtual env. It happens every time. I upgraded to python3.9.14 and got the same result:

Python 3.9.14 (main, Sep  7 2022, 23:43:29) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from secimport import secure_import
>>> os = secure_import('os', allow_shells=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/secimport/sandbox_helper.py", line 50, in secure_import
    assert run_dtrace_script_for_module(
  File "/usr/local/lib/python3.9/dist-packages/secimport/sandbox_helper.py", line 81, in run_dtrace_script_for_module
    module_file_path = create_dtrace_script_for_module(
  File "/usr/local/lib/python3.9/dist-packages/secimport/sandbox_helper.py", line 128, in create_dtrace_script_for_module
    module = importlib.machinery.PathFinder().find_spec(module_name)
AttributeError: module 'importlib' has no attribute 'machinery'

I'm not worried about this odd issue (which may just be something with my env) as I use py10 and don't have time to investigate it further right now.

seanmcfeely commented 2 years ago

Looking at the command line options you're using and comparing the man page on MacOS to the apt version:

Python 3.10.7 (main, Sep  8 2022, 01:46:46) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from secimport import secure_import
>>> os = secure_import('os', allow_shells=False, use_sudo=False)
Successfully compiled dtrace profile:  /tmp/.secimport/sandbox_os.d
(running dtrace supervisor):   dtrace -q -s /tmp/.secimport/sandbox_os.d -p 778 -o /tmp/.secimport/sandbox_os.log &2>/dev/null
/usr/bin/dtrace invalid option -q
Usage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]

It appears the -p functionality may not exist for dtrace alone via systemtap-sdt-dev but instead we'd have to use the output in combination with the Systemtap tool.

sudo apt-get install -y systemtap gcc

What do you think? Does this look like a viable route?

avilum commented 2 years ago

An eBPF/bpftrace/bcc solution should be applied to Linux based systems, which I am working on at the moment.

It seems like the systemtap and systemtap-sdt-dev are not suitable for this kind of task, they are frontend bindings / CLI tools and not dtrace itself. dtrace (for the kernel) can be installed using the dtrace4linux port, but It does not work out of the box with the current solution (DTrace for Max/Solaris) and will never have full compatibility.

Therefore, I'm working on a new backend (bpftrace) that will run also on any Linux that is supported here: https://github.com/iovisor/bpftrace/tree/master/docker

After this change, secimport will be available inside a docker container, to try over dtrace on mac, solaris. For Linux, eBPF will be used. eBPF is the most relevant solution for linux - not dtrace after all.

I will update you on the progress, I'll probably release it this month :)

seanmcfeely commented 2 years ago

Excellent!

You've got a gem 💎 in progress.

It would be wonderful to have this released this month, so let me know if you need anything. Thanks!

seanmcfeely commented 2 years ago

Hi @avilum, have you had any margin to work on this any?

avilum commented 2 years ago

Hi, I made a big progress, adding bpftrace as a backend for this project alongside dtrace.

It was a lot of code, basically implementing almost all of the templates (and unit tests) from scratch over bpftrace (the new ebpf-based backend).

I’m on vacation now (that’s why I didn’t commit in the last month), but I will commit a stale PR around the 23rd when I’m back home.

Of course it will require further testing to make sure it is stable enough, but the only requirement will be bpftrace (installable through apt-get).

It would be amazing if you could help me playing with the branch before I will merge it.

On Wed, 12 Oct 2022 at 17:22 Sean McFeely @.***> wrote:

Hi @avilum https://github.com/avilum, have you had any margin to work on this any?

— Reply to this email directly, view it on GitHub https://github.com/avilum/secimport/issues/10#issuecomment-1276357192, or unsubscribe https://github.com/notifications/unsubscribe-auth/AES2CJSG72YZZ77XMIH35TTWC3JSZANCNFSM6AAAAAAQDIR3FU . You are receiving this because you were mentioned.Message ID: @.***>

seanmcfeely commented 2 years ago

That's great news. I can absolutely help you play with the branch before merging it. Once you get the branch pushed out I will pull it and see how it does with my use case.

I did glance around bpftrace and bcc. Will python need to execute with sudo in-order to execute bptrace with sudo? I hope not.

avilum commented 2 years ago

Hey :) I opened a pull request and pushed the changes to https://github.com/avilum/secimport/pull/11 I still have to implement the tests for bpftrace backend, but you can see the code for now and maybe even use it.

I basically added

seanmcfeely commented 2 years ago

Great, thank you!

I assume it's a requirement to run python with root level permissions in-order for secimport to use bpftrace?

Is it possible to not run bpftrace as sudo if the user executing bpftrace has fill rights to the python executable?

seanmcfeely commented 2 years ago

Also, let me know if you prefer me to move my comments from the PR to this issue. If you prefer to have the conversation all in this issue.

avilum commented 1 year ago

@seanmcfeely sudo is not required unless you want destructive behavior (Killing a process upon violation of the profile). Regarding the python executable, currently this argument is overridable in secure_import and it is taken from sys.executable for bpftrace backend (the default for Linux).

I have merged the bpftrace PR #11 to master / version 0.5.0 on pypi: https://pypi.org/project/secimport/0.5.0/ Adding bpftrace support to the package. In the repo, I added docker, tests and more.

Meanwhile I'm closing the Issue, since Debian based systems are now supported. Your'e also welcome to create a PR if you have any features that I haven't fixed yet. Thanks a lot!

avilum commented 1 year ago

@seanmcfeely I have added CLI usage on debian, as well as docker container with working environments and scripts to start.

I would recommend starting with:

  1. https://github.com/avilum/secimport/wiki#docker
  2. https://github.com/avilum/secimport/wiki/Command-Line-Usage

Let me know what you think.