Restrict Mode Examples?

[VeNoMouS]

Hi @jondy,

I know your pretty busy resolving the issues that came about with 6.3.0, I was just wondering if i could get some clear proper examples of how to use the restrict modes..

What im trying to do

if i have a dir

__init__.py
packer.py

I want __init__.py to be import-able with restrict 1, but how do i then also have packer.py restrict 3 so it cant be imported outside of the __init__.py?

You doc doesn't cover proper examples things like this, it just doesn't make it clear enough , sorry

[martyur]

I thought this is what restrict 4 is for ?

[VeNoMouS]

restrict 4 allows for plain source, either way, if you set restrict 4, the init wont load.

RuntimeError: Check restrict mode of module failed

hence why im asking for proper real world examples from @jondy ;P so there is no confusion, as ... well.. there is no examples dir in the repo, and there is no proper examples in the doc.

[martyur]

You're right, my bad. I also find that the language used to describe restrict modes isn't very clear so improvement and clarification there would be very appreciated.

[VeNoMouS]

I see this... buried in the doc https://pyarmor.readthedocs.io/en/latest/advanced.html#improving-the-security-by-restrict-mode

cd /path/to/mypkg
pyarmor obfuscate --exact __init__.py exported_func.py
pyarmor obfuscate --restrict 4 --recursive \
        --exclude __init__.py --exclude exported_func.py .

[jondy]

Well, I'll refine the document after these bugs are fixed, and there are some changes for new version, which also need rewrite some of the document.

@VeNoMouS Just as you have found

The first command just obfuscates 2 scripts with mode 1 in the output dist, the second command obfuscate all the others with mode 4.

[VeNoMouS]

Yea... i dunno what im doing wrong...

python3.7 /usr/local/bin/pyarmor obfuscate --advanced 2 --platform linux.x86_64 --exact __init__.py
python3.7 /usr/local/bin/pyarmor obfuscate --advanced 2 --restrict 3 --recursive \
    --platform linux.x86_64 --exclude __init__.py .

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "</Project/Helheim/helheim/dist/__init__.py>", line 3, in <module>
  File "<frozen helheim>", line 20, in <module>
  File "</Project/Helheim/helheim/dist/packer.py>", line 2, in <module>
RuntimeError: Check restrict mode of module failed

[martyur]

The obfuscated scripts could not be imported out of the obfuscated script

All the functions in the obfuscated script cound not be called out of the obfuscated scripts.

@jondy When it refers to obfuscated script's does that mean, it can only be imported/called by obfuscated scripts that were obfuscated by any capsule or only by the same capsule ?

[jondy]

• Restrict mode 1

It's only check the form of code object.

For example, foo.py, it must be this form in the super mode

from pytransform import pyarmor
pyarmor(__file__, __name__, b'xxxx', 1)

• Restrict mode 2

When import a module obfuscated by restrict mode 2, pyarmor will first check main script, if the main script is not an obfuscated script, import failed.

• Restrict mode 3

When import a mode 3 module, pyarmor will also check the main script, if it's not obfuscated script, import failed.

And when any function in this module is called from other module, before restoring the obfuscated function, pyarmor will also check the caller, if the caller isn't in the obfuscated scripts, it will raise restrict exception.

For example, foo3.py is obfuscated by restrict mode 3, there is another script test.py, call function do_something

from foo3 import do_something
do_something()

When do_something starts, it first checks the caller module test.py, if test.py is an obfuscated script, restores the byte code and run it normally. Otherwise raises restrict exception.

• Restrict mode 4

When import a mode 4 module, pyarmor will NOT check the main script, that's different from mode 3

But when any function in this module is called, it will check the caller as mode 3.

[jondy]

The obfuscated scripts could not be imported out of the obfuscated script

All the functions in the obfuscated script cound not be called out of the obfuscated scripts.

@jondy When it refers to obfuscated script's does that mean, it can only be imported/called by obfuscated scripts that were obfuscated by any capsule or only by the same capsule ?

If the scripts are obfuscated by other capsule, it could not import your scripts at all.

[VeNoMouS]

@jondy any idea why mines failing? __init__.py and packer are both obfuscated ... i have to manually edit the obfuscated packer.py to add .for the transform

[jondy]

Set option --bootstrap 3, it will always make a relative import with leading dot.

Actually, for the super mode, the prefer way is make runtime file pytransform.so in any Python path, and always use absolute import without leading dot. Because if there are sub-folder, the relative import with leading . will fail.

[VeNoMouS]

pep8 standards states that leading . is correct also... py3.7 removing .

ModuleNotFoundError: No module named 'packer'

though this is without setting the dist to a python path...

also... adding bootstrap3 i still get if i leave import as from .packer import Packer

so in the __init__.py

from .packer import Packer

will result in...

RuntimeError: Check restrict mode of module failed

INFO     PyArmor Version 6.2.9
INFO     Target platforms: ['linux.x86_64']
INFO     Source path is "/Project/Helheim/helheim"
INFO     Entry scripts are ['__init__.py']
INFO     Use cached capsule /root/.pyarmor/.pyarmor_capsule.zip
INFO     Search scripts mode: Exact
INFO     Save obfuscated scripts to "../dist/helheim"
INFO     Read public key from capsule
INFO     Obfuscate module mode is 1
INFO     Obfuscate code mode is 1
INFO     Wrap mode is 1
INFO     Restrict mode is 1
INFO     Advanced mode is 2
INFO     Generating super runtime library to ../dist/helheim
INFO     Load platform list from /usr/local/lib/python3.7/dist-packages/pyarmor/platforms/index.json
INFO     Extract pytransform.key
INFO     Generate default license file
INFO     Copying /root/.pyarmor/platforms/linux/x86_64/11/py37/pytransform.cpython-37m-x86_64-linux-gnu.so
INFO     Patch extension ../dist/helheim/pytransform.cpython-37m-x86_64-linux-gnu.so
INFO     Generate runtime files OK
INFO     Use protection template: /usr/local/lib/python3.7/dist-packages/pyarmor/protect_code2.pt
INFO     Start obfuscating the scripts...
INFO            /Project/Helheim/helheim/__init__.py -> ../dist/helheim/__init__.py
INFO            Patch function "send" at line 253
INFO     Obfuscate 1 scripts OK.
INFO     PyArmor Version 6.2.9
INFO     Target platforms: ['linux.x86_64']
INFO     Source path is "/Project/Helheim/helheim"
INFO     Entry scripts are []
INFO     Use cached capsule /root/.pyarmor/.pyarmor_capsule.zip
INFO     Search scripts mode: Recursive
INFO     Exclude pattern "__init__.py"
INFO     Save obfuscated scripts to "../dist/helheim"
INFO     Read public key from capsule
INFO     Obfuscate module mode is 1
INFO     Obfuscate code mode is 1
INFO     Wrap mode is 1
INFO     Restrict mode is 3
INFO     Advanced mode is 2
INFO     Generating super runtime library to ../dist/helheim
INFO     Load platform list from /usr/local/lib/python3.7/dist-packages/pyarmor/platforms/index.json
INFO     Extract pytransform.key
INFO     Generate default license file
INFO     Copying /root/.pyarmor/platforms/linux/x86_64/11/py37/pytransform.cpython-37m-x86_64-linux-gnu.so
INFO     Patch extension ../dist/helheim/pytransform.cpython-37m-x86_64-linux-gnu.so
INFO     Generate runtime files OK
INFO     Use protection template: /usr/local/lib/python3.7/dist-packages/pyarmor/protect_code2.pt
INFO     Start obfuscating the scripts...
INFO            packer.py -> ../dist/helheim/packer.py
INFO     Obfuscate 1 scripts OK.

[jondy]

@VeNoMouS the packer should be obfuscated by restrict mode 4 in the package.

Mode 2, 3 is mainly for standalone scripts.

[VeNoMouS]

Ok let me try

[VeNoMouS]

While that does work... I can import packer manually...

>>> import helheim
>>> import helheim.packer
>>> helheim.packer.
helheim.packer.Packer(   helheim.packer.pyarmor(
>>> dir(helheim.packer)
['Packer', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'pyarmor']
>>>

Which should not be possible.

[jondy]

Could you call any function in the helheim.packer?

[VeNoMouS]

I think its due to the fact that packer is being imported into the __init__.py and its going __init__ -> packer, if i take it out of the init i cant import the packer.

and yes... leaving it in imported via init... all the class and functiosn are fully exposed.

so that means , you are exposed by anything that __init__.py imports.

[jondy]

Yes, this is the idea of restrict mode 4. Or write another script used to export functions which also is obfuscated by mode 1.

[VeNoMouS]

It's ok I can rewrite my code... but @jondy i hope you understand what i mean about the doco doesnt explain too well ;P

anyways dude, i just want to thank you for taking your time to respond and the work your doign..

cheers mate :)

[VeNoMouS]

Addtionally, better way to do things would be to have a shunt in the __init__.py of another script to do imports etc and that can be restrict 4. kinda hard to explain, but its cool i got an idea

[VeNoMouS]

yea that doesn't work

even if you place a stub...

ie

__init__.py -> stub.py -> realcode with imports

and you do somethng like

__init__.py

#!/usr/bin/python3
from .stub import helheim

helheim = stub.helheim

stub.py

from . import helheim

def helheim(session, response):
   return helheim.helheim(session, response)

helheim.py

from .packer import Packer

def helheim(session, response):
    ....

soon as you load __init__.py, packer becomes accessible and is exposed outside of the obfuscated code.

>>> import helheim
>>> dir(helheim)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'helheim', 'packer', 'pyarmor', 'pytransform', 'stub']

• Note: import helheim is the dir, i am not importing helheim.py

[jondy]

@VeNoMouS It's hard to hidden the module name totally, but the functions should not be called if this module is in restrict mode 4.

[VeNoMouS]

I was able to directly call them

after importing the init... i can then call functions and classes in helheim.packer.... its fully exposed there is no blocking happening like the restricted mode states.

I can access via

x = helheim.packer.someclass()
x.somefunction()

helheim being the dir

[VeNoMouS]

Was also able to replicate it in advanced mode 1, so its not supermode issue...

[VeNoMouS]

mmmm something isnt right with the restrict...

Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import helheim.packer
>>>

shouldnt this be blocking me from doing that?!

[jondy]

I'll check it later, I have restrict mode 4 test cases and they pass. But in my test cases it's import the obfuscated package from the script, not from Python interpreter, I'm not sure this.

[VeNoMouS]

cheers for looking into this for us @jondy , theres a number of users in my discord followng this thread as they use pyarmor themselves

root@tvz0r:/Project/Helheim/dist# cat test.py
#!/usr/bin/python3

import helheim.packer

root@tvz0r:/Project/Helheim/dist# ./test.py
root@tvz0r:/Project/Helheim/dist#

^^ fwiw

[VeNoMouS]

Can I ask is this wrong...

python3.7 \
 /usr/local/bin/pyarmor \
 obfuscate \
 --advanced 2 \
 --platform linux.x86_64 \
 --output ../dist/helheim \
 --exact __init__.py

python3.7 \
 /usr/local/bin/pyarmor \
 obfuscate \
 --advanced 2 \
 --restrict 4 \
 --bootstrap 3 \
 --recursive \
 --platform linux.x86_64 \
 --output ../dist/helheim \
 --exclude __init__.py \
 .

[jondy]

It seems there is a bug for importing the obfuscated package from Python interpreter directly. Yes, I also could import the obfuscated module from interpreter directly, but not for the plain script. I'll fix it with new version.

[VeNoMouS]

ah, ❤️ thanks @jondy , thought i was was doing something wrong

[jondy]

@VeNoMouS Please upgrade to latest version v6.3.1, fix restrict mode issues.

[VeNoMouS]

@jondy just got home, will try shortly for you and update , just warming up, frozen riding on the back roads at night... brrrrrr

[VeNoMouS]

Sorry mate, still can load, it didnt fix it

[jondy]

Here it's my simple testcase,

1. Package files

mypkg/__init__.py

from .bar import Testing, print_msg

mypkg/bar.py

class Testing():
    def __init__(self):
        self.key = 'ABCD'
def print_msg():
    print('This is restrict mode 4 testing')

2. Obfuscate package to dist

   pyarmor obfuscate -O dist --restrict 1 --advanced 2 mypkg/__init__.py
   pyarmor obfuscate -O dist --restrict 4 --advanced 2 --bootstrap 3 --exact mypkg/bar.py

3. Try to import dist

   
   Python 3.7.5 (default, Nov  1 2019, 02:16:32) 
   [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
   Type "help", "copyright", "credits" or "license" for more information.

import dist dir(dist) ['Testing', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'path', 'spec', 'bar', 'print_msg', 'pyarmor', 'pytransform']

dist.Testing() Traceback (most recent call last): File "", line 1, in File "", line 3, in init RuntimeError: This function could not be called from the plain script

dist.bar.print_msg() Traceback (most recent call last): File "", line 1, in File "", line 5, in print_msg RuntimeError: This function could not be called from the plain script

The functions in the `mypkg/bar.py` could not be called by outer in anyway.

[VeNoMouS]

ok let me get rid of the stub and try again.

[VeNoMouS]

@jondy but if you have another file foo.py and in bar.py...

from .foo import somefunction

foo cant be found now...

also

in python cli

import dist.bar
test = dist.bar.Testing()

^^ will work and bypass

[VeNoMouS]

Actually doesnt work on yours.. but does on my code... wtf?!

[jondy]

Really? I'll check it.

[VeNoMouS]

I need to pull this apart and work out why yours works and mine doesnt...

[jondy]

No, it still can't be called.

So make sure the scripts are obfuscated by right mode and not be overwritten, check the console output which will report which scripts are obfuscated by this command.

And make sure the dynamic library is latest.

[VeNoMouS]

ok... i think i discovered it hold up..

[VeNoMouS]

100% confirmed

it seems you can bypass on classes... not functions...

>>> import dist.foo
>>> dir(dist.foo)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'blah', 'pyarmor', 'test']
>>> dist.foo.test()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen foo>", line 2, in test
RuntimeError: This function could not be called from the plain script
>>> dist.foo.blah()
<dist.foo.blah object at 0x7fb2781215f8>

aahhh... however.. you cant execute functions inside from the class..

>>> dist.foo.blah().blahtest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen foo>", line 6, in blahtest
RuntimeError: This function could not be called from the plain script
>>> x = dist.foo.blah()
>>> x.blahtest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen foo>", line 6, in blahtest
RuntimeError: This function could not be called from the plain script
>>>

[VeNoMouS]

init.py

from .bar import Testing, print_msg

foo.py

def test():
    print('made it here')

class blah():
    def blahtest(self):
        print('made it here in blahtest..')

bar.py

from .foo import test

class Testing():
    def __init__(self):
        self.key = 'ABCD'

def print_msg():
    print('This is restrict mode 4 testing')

[VeNoMouS]

>>> import helheim.packer
>>> dir(helheim.packer.Packer())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen helheim.packer>", line 5, in __init__
RuntimeError: This function could not be called from the plain script
>>>

sorted it

as long as you have a __init__ in the class, it doesnt expose the functions in the class.

[VeNoMouS]

ah... my bad if you dont create a pointer to the object function and call the attritube name..

>>> dir(helheim.packer.Packer)

it still exposes.

[VeNoMouS]

using the example i posted before...

Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dist.foo
>>> dir(dist.foo.blah())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen foo>", line 6, in __init__
RuntimeError: This function could not be called from the plain script
>>> dir(dist.foo.blah)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'blahtest']
>>>

it only exposes function names... cant execute them

[VeNoMouS]

I could always do closures to hide...

[VeNoMouS]

nope... still exposed in a closures... lol its ok ill take this off topic with myself to resolve, thanks for your help in resolving those other things though @jondy <3

get that damn donate page up ;P

[VeNoMouS]

hrrm seems it isnt explosed in a closure

[VeNoMouS]

@jondy heads up...

When calling a function from a restricted module, it errors as it should, however upon exiting python or even doing anything else in python, it produces a segfault

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen helheim.core>", line 446, in helheim
RuntimeError: This function could not be called from the plain script
>>>
Segmentation fault