[Feature request] option to encrypt without obfuscation

juanlao7 / codeclose

A license system and code obfuscator for Python.

https://juanlao7.github.io/codeclose

MIT License

23 stars 3 forks source link

[Feature request] option to encrypt without obfuscation #1

Open LaurentEsingle opened 3 years ago

LaurentEsingle commented 3 years ago

Hi,

First of all thank you for your great work and your decision to publish that work so that the community can benefit from it.

I think it would be beneficial to have an option to not obfuscate some files similar to what we have with encryption exclusions.

codeclose \
    --encryption-excluded [...] 
    --obfuscation-excluded [...]

Here's why. In my particular scenario, I have a lot of modules/classes that use many third parties modules. Which means I end up with an extremely long list of identifiers/attributes to exclude. I have tried to obfuscate my entire application without success. In my case it is a daunting task.

Even if I succeeded, it would be difficult to maintain such a list with an application that gets updated constantly.

In contrast, encryption only requires a small effort to implement, and is not dependent on code change. And the performance is also great.

As a workaround, I changed the code to only obfuscate the application entry point (main.py) and encrypt the remaining files.

In model.py:

with open(srcFilePath, 'r', encoding='utf-8') as handler:
    content = handler.read()
    content = _remapModules(content, codecloseModuleName)
    if srcFilePath == "./main.py":
      print("obfuscating file: " + str(srcFilePath))
      content = obfuscator.obfuscate(content)

But I am facing two caveats with that workaround:

I have to put all my files in the same directory as main.py (which means I end up mostly with one directory with dozens of files)
There is one file that I cannot obfuscate/encrypt, which handles the security of my application and uses similar libraries as codeclose (base64 and cryptography 3.0). It also uses "decrypt" keyword in its code. This is the only notable difference I can see that sets it apart from other files.

Here are the details of the error:

~/.local/lib/python3.8/site-packages/vquery/GLOBAL/_Warning_ReferenceError/runtime.py in map_ord(divmod_help, compile_bytes, breakpoint_PendingDeprecationWarning)
     23   return ''
     24  IndexError_FloatingPointError = nonlocal_SystemExit.new(license[(3097986357711097591446599161849 + int(4938221631556028608960218434944)).to_bytes(13, 'big').decode()], nonlocal_SystemExit.MODE_CBC, iv=exit_quit(compile_bytes))
---> 25  object_BytesWarning = IndexError_FloatingPointError.oct_UnicodeEncodeError(exit_quit(divmod_help))
     26  return object_BytesWarning[:breakpoint_PendingDeprecationWarning].decode((148091511572 + int(356372515364)).to_bytes(5, 'big').decode())
     27 def license_zip(type_ChildProcessError=True):

AttributeError: 'CbcMode' object has no attribute 'oct_UnicodeEncodeError'

I think the error happens here (in runtime.py)

def expose(encryptedContent, initializationVector, originalSize):
    try:
        license = getLicense()
    except:
        return ''

    cipher = AES.new(license['encryptingKey'], AES.MODE_CBC, iv=b64decode(initializationVector))
    contentBytes = cipher.decrypt(b64decode(encryptedContent))
    return contentBytes[:originalSize].decode('utf-8')

I am not sure how to fix such an error but again thank you for letting us use your code.

juanlao7 commented 3 years ago

Hi,

Thank you for using my tool, I'm glad you find it useful! Also thanks for your feedback and your request to improve it.

Regarding the feature request, I would like to know what problems it is causing exactly. Please could you post a couple of examples of things Codeclose should not obfuscate? The fragment of the original source code and the result would be very useful to me.

Thanks!

EDIT: just to clarify, using external attributes and functions shouldn't be a problem. Codeclose only replaces identifiers that are defined or assigned, but not only read.

LaurentEsingle commented 3 years ago

Well, I had to change my scripts, folders structure etc. to test the obfuscation again. I spent a few hours trying to make it work but I eventually had to stop and show you the errors I am getting.

1) Imports aliasing:

# imports with dots cause an invalid syntax. Example below with --name-obfuscation light. But this does not seem to be an issue with full obfuscation.
import os.path as os.path_55575

# aliasing of * causes an invalid syntax:
from gremlin_python.process.strategies import * as UnicodeError_BytesWarning

2) The following syntax/commands generate an error: "AttributeError: 'Unparser' object has no attribute '_fstring_Call'". Full stack error at the end of my comment.

html = html.format(payload=payload,title=title+f' {short_file_name}',filename=logfile)

It is quite difficult for me to find what is causing issues as I cannot set break points in my Dev environment. I have to rely on tracing (logs) which are useless with obfuscation. Which is another plus in my opinion for encryption.

Today - with full obfuscation - I reached a point where I was able to launch the application and get the login screen but not beyond. In the logs I can see in which module the application stops but I have no idea what is going on. And I get nowhere with obfuscation light as it requires me to change code in many places to solve import issues. I will continue debugging tomorrow and let you know.

Error stack:



model.protect(args.encrypting_key_path.read(), args.dest_directory, args.src, args.encryption_excluded, args.keep_identifier, args.keep_attributes, args.name_obfuscation, not args.disable_string_obfuscation, args.disable_encryption, args.follow_symlinks)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/codeclose/model.py", line 97, in protect
    content = obfuscator.obfuscate(content)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/codeclose/obfuscation.py", line 112, in obfuscate
    obfuscatedContent = astunparse.unparse(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/__init__.py", line 13, in unparse
    Unparser(tree, file=v)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 38, in __init__
    self.dispatch(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 78, in _Module
    self.dispatch(stmt)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 343, in _ClassDef
    self.dispatch(t.body)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
    self.dispatch(t)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 347, in _FunctionDef
    self.__FunctionDef_helper(t, "def")
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 365, in __FunctionDef_helper
    self.dispatch(t.body)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
    self.dispatch(t)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 248, in _Try
    self.dispatch(t.body)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
    self.dispatch(t)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 120, in _Assign
    self.dispatch(t.value)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 727, in _Call
    self.dispatch(e)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 844, in _keyword
    self.dispatch(t.value)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 686, in _BinOp
    self.dispatch(t.right)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 465, in _JoinedStr
    self._fstring_JoinedStr(t, string.write)
  File "/home/wsluser/Venvs/vQuery_env/lib/python3.7/site-packages/astunparse/unparser.py", line 490, in _fstring_JoinedStr
    meth = getattr(self, "_fstring_" + type(value).__name__)
``

LaurentEsingle commented 3 years ago

EDIT: just to clarify, using external attributes and functions shouldn't be a problem. Codeclose only replaces identifiers that are defined or assigned, but not only read.

Good to know. Thank you!

juanlao7 commented 3 years ago

Many thanks for the report!

I created 3 new issues to address them. I will try to fix them ASAP.

After that I will add two new options: --disable-obfuscation and --obfuscation-excluded FILE_PATH

LaurentEsingle commented 3 years ago

Thank you!

juanlao7 commented 3 years ago

Hi!

I just fixed all the reported bugs, plus added a new option --obfuscation-excluded FILE_PATH that prevents Codeclose from obfuscating the identifiers that are defined exclusively in the given file. If that file uses identifiers defined in another obfuscated file, Codeclose will still replace all their references by the obfuscated name everywhere.

I'm considering not implementing --disable-obfuscation after all, because then it would be quite trivial for malicious users to bypass all protections.

Please, let me know if that works for you ;)

Ref: 23049402b0965563ad3a661421b3c36910e68365

LaurentEsingle commented 3 years ago

Hi!

Thank you, that was quick.

I tested the new code today - although not the disable obfuscation option - and have a few observations.

1) Modules import:

# The following module import (unobfuscated code) 
import vquery.helpers.version as version

# is obfuscated as:
import vquery.helpers.version as version_81789

# and is later used as follows:
self.settings_dict["version"] = version.ver

# which is obfuscated as:
self_54367.settings_dict_54729['version'] = version_81789.ver

# Strangely enough the python interpreter (v.3.7) does not trigger an error. The program simply stops.
# The problem is easily solved with: --keep-identifier version

2) The following code snippets also cause issues:

# unobfuscated code:
self.__pager = Pagination(total=self.__total_rows, per_page=self.__rows_per_page, current_page=1)
...    
if self.__pager.current_page == len(self.__pager.pages):    
...    
self.__pager.current_page = len(self.__pager.pages)

# obfuscated version:
self_24543.__pager_81560 = Pagination(total=self_24543.__total_rows_11439, per_page=self_24543.__rows_per_page_11293, current_page=1)
...
if (self_24543.__pager_81560.current_page == len(self_24543.__pager_81560.pages_62470)):
...
self_24543.__pager_81560.current_page = len(self_24543.__pager_81560.pages_62470)

# the two last lines of code cause the program to stop its execution...
# the following exclusions did not help 
--keep-identifier __pager --keep-attributes __pager
--keep-identifier self.__pager --keep-attributes self.__pager

I will make additional tests and let you know.

juanlao7 commented 3 years ago

Thanks for the report!

I've been checking the new cases.

I think the program stops without displaying an error because there's a try-except somewhere catching and ignoring the exception. Is that possible?
Regarding case 1, I don't understand why it fails. The obfuscated alias "version_81789" is correctly referenced later, it shouldn't fail. Getting the exception thrown for this one would be very useful.
Regarding case 2, you found a bug. Seems that --keep-attributes work in the form "keep_these.x", but not in the form "x.keep_these.x". I just fixed that in 100cd3816cac1e7b189ff2c3f585c73ab53709cb

Please let me know if you find something else!

LaurentEsingle commented 3 years ago

Hi! Thank you for your changes.

I have been busy these last days updating my environments to python3.8 (from python3.7) only to find out that codeclose is not compatible with python3.8

The issue is due to errors in astunparse library. Full error stack below.

For more information on the issue, please visit this page (read until the end): https://github.com/simonpercivall/astunparse/issues/43

Now I am busy reverting to python3.7.5 and will give you a feedback about your latest changes shortly.

Traceback (most recent call last):
  File "/home/wsluser/.local/bin/codeclose", line 8, in <module>
    sys.exit(main())
  File "/home/wsluser/.local/lib/python3.8/site-packages/codeclose/__main__.py", line 196, in main
    args.handler(args)
  File "/home/wsluser/.local/lib/python3.8/site-packages/codeclose/__main__.py", line 110, in handler
    model.protect(args.encrypting_key_path.read(), args.dest_directory, args.src, args.encryption_excluded, args.keep_identifier, args.keep_attributes, args.name_obfuscation, not args.disable_string_obfuscation, args.disable_encryption, args.follow_symlinks)
  File "/home/wsluser/.local/lib/python3.8/site-packages/codeclose/model.py", line 76, in protect
    _injectContent(injectionContent, os.path.join(destPath, injectedCodeclosePackageName), obfuscator)
  File "/home/wsluser/.local/lib/python3.8/site-packages/codeclose/model.py", line 174, in _injectContent
    content = obfuscator.obfuscate(injectionContent[injectedFilePath])
  File "/home/wsluser/.local/lib/python3.8/site-packages/codeclose/obfuscation.py", line 112, in obfuscate
    obfuscatedContent = astunparse.unparse(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/__init__.py", line 13, in unparse
    Unparser(tree, file=v)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 38, in __init__
    self.dispatch(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 78, in _Module
    self.dispatch(stmt)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 347, in _FunctionDef
    self.__FunctionDef_helper(t, "def")
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 365, in __FunctionDef_helper
    self.dispatch(t.body)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 63, in dispatch
    self.dispatch(t)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 120, in _Assign
    self.dispatch(t.value)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 723, in _Call
    self.dispatch(e)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 744, in _Subscript
    self.dispatch(t.slice)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 756, in _Index
    self.dispatch(t.value)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 717, in _Call
    self.dispatch(t.func)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 707, in _Attribute
    self.dispatch(t.value)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 717, in _Call
    self.dispatch(t.func)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 707, in _Attribute
    self.dispatch(t.value)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 684, in _BinOp
    self.dispatch(t.left)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch
    meth(tree)
  File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 551, in _Constant
    if t.kind == "u":
AttributeError: 'Constant' object has no attribute 'kind'

LaurentEsingle commented 3 years ago

Hi!

Good news! The application is working with obfuscation and encryption. I excluded all files from obfuscation (except the entry point) using "--obfuscation-excluded" flag.

Not everything is working as I still have to exclude some identifiers here and there and refactor the code where needed, but it almost done. Thank you!

The only thing left (in my opinion) is the python3.8+ bug I described earlier. I need to upgrade to python3.8 unfortunately.

Again thank you. If if find anything else I will let you know. Still testing the application.

juanlao7 commented 3 years ago

Hi!

I'm glad it works!

I will check the Python 3.8 issue. Maybe I can replace the constant by something else in the tree before running astunparse.

Thanks!!

juanlao7 commented 3 years ago

Hi Laurent,

Happy new year! Sorry for the delay, I was a little bit busy during the holidays.

I'm trying to reproduce the problem in Python 3.8.7, but I can't find a way of making it crash. I thought it was as simple as writing a u'unicode string' in the code I'm obfuscating.

Please could you give me an example of code that makes the system crash?

LaurentEsingle commented 3 years ago

Hi Juan,

I was not aware you posted this last comment! Just saw it now, almost two months later...

So you do not have the issue on Python 3..8.7 ? I am using Python 3.8.5

I will test again with Python 3.8.5 and then Python 3.8.7 and let you know.

Thank you for your efforts. Your project is very helpful to me.

Laurent.

LaurentEsingle commented 3 years ago

Hi! @juanlao7

I am not able to reproduce the "constant" error either, at least for now. The code has substantially changed since last December and that could be the reason.

But I cannot reach a definitive conclusion as I am facing new issues that prevent me from using the obfuscated application.

Codeclose options used for the sample code below are:

--disable-encryption
--disable-string-obfuscation
--name-obfuscation light
--keep-identifier tmp_dict2 --keep-attributes tmp_dict2
--keep-identifier tmp_lst --keep-attributes tmp_lst

Code:

tmp_dict = {'name': 'value'}
tmp_dict2 = tmp_dict.copy()

tmp_lst = ['one', 'two']
tmp_lst.sort()

Obfuscation:

tmp_dict_25250 = {'name': 'value'}
tmp_dict2_55191 = tmp_dict_25250.copy_42835()
tmp_lst_98502 = ['one', 'two']
tmp_lst_98502.sort_21163()

Errors

AttributeError: 'dict' object has no attribute 'copy_42835'

AttributeError: 'list' object has no attribute 'sort_21163'

I am not sure I had these issues before the last Codeclose code change. But I could be wrong. It was almost two months ago.

Thank you for your help.

Laurent.

juanlao7 commented 3 years ago

Hello Laurent,

This is weird, I can't reproduce that problem either.

I committed a new example case similar to yours. You can compile it by just running bash protect.sh example7 inside directory examples. You will find the results inside directory examples/protected/example7/.

I tried to manually use the same arguments you are using and it still works. I am using Python 3.8.7 on Windows.

Does that example case work for you?

LaurentEsingle commented 3 years ago

Hi @juanlao7

Sorry for the delayed answer.

The good news is I managed to get the (full) obfuscation working on my project. Unfortunately the encryption does not work anymore.

Here are some findings and workarounds I want to share.

1) Test with Example7 I could not run that code as I get the following error (more on that later):

File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 551, in _Constant
    if t.kind == "u":
AttributeError: 'Constant' object has no attribute 'kind'

2) Solution to the attribute error The problems I faced:

tmp_dict_25250 = {'name': 'value'}
tmp_dict2_55191 = tmp_dict_25250.copy_42835()
tmp_lst_98502 = ['one', 'two']
tmp_lst_98502.sort_21163()

were due to having somewhere else in my code:

import copy
...
sort = "df = df....."

which was obfuscated as:

import copy as copy_42835
...
sort_21163 = "df = df....."

what it means is if you have an import or variable which as the same name as an identifier's attribute you will get the result above, even if you try to avoid it using --keep-identifier and --keep-attributes flags.

3) Astunparse issue As we discussed before, some features of astunparse library are broken in python 3.8+ (works well in python 3.7). When string obfuscation is activated (meaning --disable-string-obfuscation flag) we get the following error:

File "/home/wsluser/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 551, in _Constant
    if t.kind == "u":
AttributeError: 'Constant' object has no attribute 'kind'

The workaround I found was to downgrade to version 1.6.2 from 1.6.3 which is automatically installed as a dependency of codeclose:

pip install --upgrade astunparse==1.6.2

Which makes me wonder if this is why you are not getting the error. Are you using version 1.6.2?

4) Encryption Encryption used to work in python 3.7 for me but does not work anymore with python 3.8. And I cannot find why. I get the following error when trying with full obfuscation:

~/.local/lib/python3.8/site-packages/vquery/_chr_hex/runtime.py in else_all(StopAsyncIteration_set, loader_FutureWarning, EnvironmentError_DeprecationWarning)
     24  await_in = not_ConnectionError.new(license[(2268948717979346281280534972561 + int(5767259271287779919126282624232)).to_bytes(13, 'big').decode()], not_ConnectionError.MODE_CBC, iv=ResourceWarning_GeneratorExit(loader_FutureWarning))
     25  DeprecationWarning_SystemError = await_in.decrypt(ResourceWarning_GeneratorExit(StopAsyncIteration_set))
---> 26  return DeprecationWarning_SystemError[:EnvironmentError_DeprecationWarning].decode((363633351360 + int(140830675576)).to_bytes(5, 'big').decode())
     27 def help_open(with_SystemExit=True):
     28  global _settings

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte

Which seems to happen here (in runtime.py):

cipher = AES.new(license['encryptingKey'], AES.MODE_CBC, iv=b64decode(initializationVector))
contentBytes = cipher.decrypt(b64decode(encryptedContent))

I have tried to encrypt with any combination of the following flags (--disable-string-obfuscation, --name-obfuscation light) without success.

5) Suggestion When I look at the obfuscated code in the entry module (main in my case), codeclose arguments are... too visible:

StopIteration_RecursionError(verifyingPublicKey=lc, encryptingKey=mu, expectedProductIds=[2], productKey=st)

maybe we could use something like this?

StopIteration_RecursionError(vp=lc, ek=mu, prd=[2], kp=st)

6) Thank you Codeclose is very useful and one of the best implementation I have found. It requires a little bit of work to exclude identifiers and attributes - especially if the codebase is large - but the effort is worth it.

I saw somewhere in your code that a license server can be used. If it is not too difficult and you can guide me with the implementation I might try to do it in the future (in a month or so) as my code runs in a container and a license server is recommended.

Thanks!

juanlao7 commented 3 years ago

Hi Laurent!

Thank you very much for your feedback, it is really useful for finding and fixing the cause of the problems. I am glad Codeclose is useful for you!

Which makes me wonder if this is why you are not getting the error. Are you using version 1.6.2?

Seems that this problem doesn't occur on Windows (Python 3.8.7, Astunparse 1.6.3). However, I could reproduce it on Linux (Python 3.8.4, Astunparse 1.6.3).

Since Astunparse won't fix the issue, I made a workaround and fixed the problem in commit 0f145f53a3253f9c593c537e1a25c9ef445f30d1

Please, let me know if you can still reproduce the problem.

what it means is if you have an import or variable which as the same name as an identifier's attribute you will get the result above, even if you try to avoid it using --keep-identifier and --keep-attributes flags.

I am trying to reproduce the problem, but everything works fine in my environment.

I committed a new example case similar to your case. You can compile it by just running bash protect.sh example6 inside directory examples. You will find the results inside directory examples/protected/example6/.

This is the source code:

# Example to reproduce https://github.com/juanlao7/codeclose/issues/1#issuecomment-814175447

# These identifiers should be obfuscated.
import copy
sort = 1

# But the attributes of this object should be kept.
attributesKept = [2, 1]
attributesKept.copy()
attributesKept.sort()

And this is the obfuscated result:

import copy as IsADirectoryError_ReferenceError
break_compile = 1
attributesKept = [2, 1]
attributesKept.copy()
attributesKept.sort()

I think the problem in your case is that the identifiers tmp_dict2 and tmp_lst are getting obfuscated (and that's the reason why the system is not keeping their attributes). I think running Codeclose with options --keep-identifier tmp_dict2 and --keep-identifier tmp_lst should fix the problem.

Encryption used to work in python 3.7 for me but does not work anymore with python 3.8. And I cannot find why. [...] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte

Seems that the problem is that, after decrypting one of your files, Python cannot convert the decrypted bytes into an UTF-8 string because the first character/byte is 0xfe.

If none of your source files contain this character, then the problem may be that, by mistake, you are trying to decrypt with a different encryption key than the one you used for encrypting. Can that be the case?

If that's not the case, I would really appreciate if you could send me a little example of code that triggers this problem, so I can look into it.

When I look at the obfuscated code in the entry module (main in my case), codeclose arguments are... too visible:

Thanks for the feedback! I just fixed it in commit 901dd525a7e479aaa9dc8e83e34b8b041c58c135

However, take into account that Codeclose does not obfuscate keyword arguments (a.k.a. kwargs), because usually the function access them by the string key (e.g., kwargs['param1']), and the obfuscator wouldn't be able to convert the keys into their obfuscated versions.

If you want to obfuscate the parameters passed to configure(...), now you can pass them as positional arguments in the following order:

configure(productKey, verifyingPublicKey, encryptingKey, expectedProductIds)

I saw somewhere in your code that a license server can be used. If it is not too difficult and you can guide me with the implementation I might try to do it in the future (in a month or so) as my code runs in a container and a license server is recommended.

I guess you are referring to runtime.py:getLicense(), the line that says:

# TODO: obtain the license from the remote server

At this moment the license is being computed locally, and that's why you need to store and provide the verifying public key and the encrypting key.

This allows you to validate product keys offline, without requiring an internet connection. However, this is a weak point for Codeclose, because someone can debug your app to extract the encrypting key and decrypt all your code.

My plan was to implement some kind of optional online validation. Instead of configuring Codeclose with the verifying and the encrypting key, you could configure it with the URL of a validating server, and then Codeclose would make an HTTP request to send the product key. If the product key is valid, the server would respond with the encrypting key, and then Codeclose would be able to decrypt and run all the code.

The bad news is that this feature is not implemented right now (and unfortunately I don't have plans to implement it in the short term).

The good news is that you can implement a similar system independently of Codeclose:

Set up a public server with a Python backend. Whenever you receive a product key through HTTP, check if it is valid with Codeclose and return the secret encrypting key if it is so.
In the app, before calling configure(...), make a request to your server and send the user's product key. Then, after you receive the encrypting key from the server, call configure(...) with it, as usual.

I hope this helps!