search

dashingsoft / pyarmor

A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts.
http://pyarmor.dashingsoft.com
Other
3.35k stars 283 forks source link

dramatiq and http server #193

Closed evstratbg closed 4 years ago

evstratbg commented 4 years ago

Hi! Thanks for great work. We are using pyarmor in our company and very happy with it. But there are some buts ツ

We have a simple http server, written in flask, receiving tasks and put then into queue. Also we use dramatiq to get tasks from queue done.

I'm stucked with obfuscating this project. The project structure is:

dramatiq_app/
├── __init__.py
├── tasks/
│   ├── __init__.py
│   ├── get_files
│   │   ├── __init__.py
│   │   ├── base.py
│   │   ├── content_service.py
│   │   ├── another_service.py
app
├── __init__.py
├── handlers/
│   ├── pipelines.py

run.py

So, to obfuscate server, I just run pyarmor obfuscate -r run.py and everything is fine. But when I try to run mq worker like dramatiq dramatiq_app.tasks.get_files , I'm mostly getting this error

[2020-02-20 14:06:40,274] [PID 454] [MainThread] [dramatiq.MainProcess] [INFO] Dramatiq '1.7.0' is booting up.
Process Process-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/site-packages/dramatiq/cli.py", line 301, in worker_process
    module, broker = import_broker(args.broker)
  File "/usr/local/lib/python3.6/site-packages/dramatiq/cli.py", line 107, in import_broker
    module = importlib.import_module(modname)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "./dramatiq_app/__init__.py", line 1, in <module>
    __pyarmor__(__name__, __file__, b'\x50\x59\x41\x52\x4d\x4f\x52\x00\x00\x03\x06\x00\x33\x0d\x0d\x0a\x02\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x40\x00\x00\x00\xd5\x03\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf4\x89\x4a\x66\x47\x21\x2d\xd2\x6f\xab\x80\x27\xb7\x93\xf6\x76\x5f\x9a\xf3\x9a\xd2\x7e\xfe\x3f\xc9\xc8\xad\xa6\xf5\xeb\xde\xdc\x73\x6d\x2b\x16\xe5\xbd\xfd\x44\xc8\xc5\x92\xda\xda\x0a\xe5\x1a\xbf\x56\x08\x14\x65\x78\x28\x98\x53\x05\xda\xcb\x20\x54\x81\x8d\xdc\xd5\x7a\x39\x39\xe2\x51\xce\x61\xb7\x70\x28\xc4\x2d\xf9\x33\x8b\x56\x24\x5d\xdf\x47\xd5\x12\xef\x04\x4d\xac\xa2\x18\xe9\x25\x1a\xb2\x9e\x25\xdd\xe8\xc4\x77\x62\xed\xf0\xfb\x3f\xe4\xb4\x67\xf8\x17\x5b\x5d\x1b\x61\x53\x35\xf4\x0d\xe0\xdf\xe8\x37\xda\x86\x1e\xb8\x21\x6d\xc3\x26\x62\xea\xbf\xf0\xa7\xcd\xc9\xf8\xf3\xb8\x3c\x31\x7b\x26\xfc\x6c\x91\x5e\x03\x12\x17\xd0\x26\x45\xcf\x58\x0f\x2e\x37\xad\x64\x8b\xd2\xdd\x99\xe0\xd8\x3e\xb2\x29\xde\x2c\xbb\x64\xa9\xf6\xa3\xc7\xf4\x13\x1f\xf7\x9a\xeb\x6a\x6e\x05\xe4\x35\x64\xcc\xb9\xc2\x35\xe0\x92\xc8\xa0\x13\x14\xcf\x5d\xa6\x3f\x84\xf5\x73\x59\x3a\x57\x74\x44\x70\xdf\x9e\xf9\x48\xff\xaa\x71\x02\x9d\x82\xe6\x53\x9e\xbd\xf7\x99\x67\x82\x35\x84\xd3\x3f\x0e\x10\x93\x73\x50\x63\x4a\x62\x53\x1e\x0f\x53\xae\xc7\x2b\x52\x8c\xad\x55\x2b\x6d\xac\x83\x07\x4c\xcc\x58\x30\xdb\x79\xf3\x9f\xd4\x23\x14\xa1\x7d\x4c\xe9\xb7\x02\x18\xbd\x2a\x6b\xf7\xc1\x56\x3e\xc0\xec\x88\xa7\xbc\x1b\xdf\xa6\xae\x56\xfd\x0a\x9a\x41\x84\x5d\x4e\xdd\x3f\x73\x8b\x41\x0a\x54\x51\xfd\x14\x66\x4a\xb9\x06\xf7\x13\x3f\xfe\x0e\xe5\x02\x8d\x0d\xe9\xa8\x9c\xef\x4d\x2b\x21\x3c\x8c\xa9\x89\x16\xf3\x14\x29\xd9\xa9\x42\xfd\xef\x35\xcd\xb7\xe7\xef\xd6\x28\x15\x8c\x70\x77\x90\x14\x30\x1a\x72\x49\x8e\x94\xdf\x7b\x6a\x2c\xbe\x0b\x14\xb0\xad\xfa\x8a\x3d\x25\xd2\x41\x3b\x2a\x8d\xf6\x62\xb7\x6f\x64\x4b\xd8\x44\xdb\x6a\x82\x2a\x6f\xb1\x09\x91\xfd\x8e\xc6\xbf\x29\x57\x86\xdd\xcf\xc0\xf2\x77\x91\xa2\x07\x70\xc6\x9d\x2b\xf8\x21\xf0\x3a\x09\x2d\x44\xe3\x03\x3f\xf9\xef\xc4\x0c\xdc\x5e\x7f\xd2\x6f\x5c\x53\x36\x5d\x06\x12\x2e\x69\x82\x88\x81\x4e\xbf\xc1\x05\x92\xbb\xa5\xe9\xf6\x79\x4c\x96\xdb\xd3\x2a\x01\xa9\x67\x7b\x5d\xe1\x7c\xdc\x6b\x7b\x8d\xbf\xb1\xd8\x5f\xfd\xac\xeb\x14\x5d\x9c\x81\xba\x1e\x90\x85\xa3\xde\x71\x5f\x2a\xcf\x20\x87\xf5\x02\xe8\x4a\x5a\x6f\x5a\x7f\x6a\x87\x32\xcd\x28\x14\x43\x10\xc1\x21\xe1\xb2\xe6\xec\xf2\x38\x87\xaf\xbd\x61\x11\xd2\x56\x61\x0d\x34\x48\xe4\x6d\x94\x5c\xa9\xe6\xc9\x7e\xb8\x93\xea\x7f\x6b\xdc\x51\x23\x8b\xad\xa4\x66\x49\x44\x37\xe5\xe9\x7d\xe6\xaa\xf9\x78\xda\xc7\x44\x00\x2a\x37\xbb\x62\x92\x89\xe2\x79\xb7\x32\x98\x26\x13\x1d\xf7\x25\xd4\x35\xb7\x82\x35\x81\x86\xd6\x8d\x86\x7a\x58\x49\x1a\x14\xc9\x81\x4a\x93\xe2\xce\xa4\xed\x9f\xaf\xc3\xc7\x72\x45\xf1\x75\x3c\x36\x86\xd7\x08\xae\x22\xb3\x1e\x28\x23\xa8\xc5\x7d\x48\x60\xd5\x00\xaa\x28\x7d\xf1\x5a\x19\x7f\x98\xb8\x8a\xf8\xae\x33\x94\x2f\x93\x29\x1e\x94\xb7\xcc\x5f\x84\xcb\xb1\xa8\x77\x51\xa0\x7c\x61\x1b\xda\xea\xc7\xd4\xb3\x0d\x64\xab\x9c\xc7\x3c\xa2\x3c\xf9\x84\xb3\x08\x47\x08\x08\xbb\xa5\x58\x61\xb3\x75\x79\x2f\x64\x6f\xa0\xe3\xfe\x47\xf9\x17\xe3\xa3\xb6\x38\x6e\x28\x06\x87\x42\xdd\xfd\x24\xa5\x23\x38\xb5\xd0\x52\xdb\xa8\x92\x93\xde\xd0\x57\xbc\xd3\xe8\xf4\x23\xb2\xef\xf5\xde\x4a\x1d\xbf\x90\x52\x6c\x0a\x76\x26\xae\xc2\xc4\x07\x40\x32\xf6\x82\xa6\x44\x54\x54\xea\xbe\x3f\xa8\xf1\xb8\x2e\x09\x2a\xbe\x2d\x71\xb8\x2c\x40\xa1\x6c\xea\x7e\x9d\xa7\x0e\x36\xd9\x99\x4c\x5b\x48\xd6\x15\xdb\xe4\x1a\xff\x33\xb6\xb6\x8d\x36\x1f\xf2\xc9\x63\xc8\xb7\xe7\x61\xe3\x8f\xc0\x77\x5d\xdb\x2c\xb4\x87\x73\x01\xe7\x85\xcb\x02\x44\xae\x23\x6d\x26\x69\xa0\xce\x78\xd5\x8e\xe7\x21\xdb\xfc\xaf\x13\x89\x09\xf9\x81\x6f\x23\x0c\x0c\x01\xdd\x7a\x89\xec\x2d\x96\x4b\xc2\xb2\x91\xab\x27\x53\x64\x69\x39\xb6\x7e\xe3\xc4\x61\x48\xdc\x16\x42\x6d\xdf\x53\x28\x7e\x20\x04\xf2\x3d\xd8\xf9\x0e\x6d\xf8\x5d\x98\x28\xc8\xe1\xe5\x82\xb0\xda\xa8\xae\x36\x2d\x2d\xd4\x27\xcb\xdf\x0d\xb9\x92\xe8\x2c\x46\x4d\x5e\x43\x8b\xd3\x6d\x91\xa5\xd8\x5c\x11\x0a\x2e\xf4\x29\x30\xf7\x20\xd7\xab\xf9\xcd\x25\xcb\x8a\x18\xa0\xd5\x96\x5e\xbf\x93\x49\xb1\x57\x2d\x12\x53\xf0\x89\x61\x6a\x07\x07\xa2\xdc\x08\xdb\xc4\xa7\xc4\xc6\xae\xb2\x80\xd9\x6c\xbe\x35\x96\x54\x96\x90\xc2\x60\x3c\xf6\x26\x82\xda\x2a\x95\xe3\xae', 1)
NameError: name '__pyarmor__' is not defined

Or stripped part of It. I guess, because I pointed run.py as entry script, but run another one.

So the questions are: 1) Should I use docs at https://pyarmor.readthedocs.io/en/latest/advanced.html#obfuscating-many-packages? 2) In example above, there is 1 entry script at dramatiq_app - get_files. In real, we have tens of them. Should I pyarmor them as separate packages?

jondy commented 4 years ago

When starts a fresh python interpreter process by multiprocssing.Process, os.exec, subprocess.Popen etc., make sure the bootstrap code are called in new process before running any obfuscated script.

  1. Should I use docs at https://pyarmor.readthedocs.io/en/latest/advanced.html#obfuscating-many-packages?

No. Instead refer to https://pyarmor.readthedocs.io/en/latest/advanced.html#let-python-interpreter-recognize-obfuscated-scripts-automatically

  1. In example above, there is 1 entry script at dramatiq_app - get_files. In real, we have tens of them. Should I pyarmor them as separate packages?

In above example, the entry script may be task/__init__.py, it could be obfuscated as entry script with option --exact

pyarmor obfuscate --no-runtime --exact task/__init__.py

If there are many entry scripts, obfuscating them by this way

pyarmor obfuscate --no-runtime --exact task/__init__.py task2/__init__.py ...

Refer to https://pyarmor.readthedocs.io/en/latest/man.html#obfuscate

evstratbg commented 4 years ago

Thanks! Links gave more understanding and everything works fine now