ClaudiuGeorgiu / Obfuscapk

An automatic obfuscation tool for Android apps that works in a black-box fashion, supports advanced obfuscation features and has a modular architecture easily extensible with new techniques
MIT License
1.11k stars 288 forks source link

Android Studio -specific ERROR: No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk' #110

Closed skreenr closed 3 years ago

skreenr commented 3 years ago

I have used Obfuscapk successfully before on APKs build with Eclipse. Having used Android Studio to build a recent APK I see strange behavior with an error:

Here is command as invoked: python -m obfuscapk.cli -o FieldRename -o Reorder -o CallIndirection -o Rebuild -o NewSignature -o NewAlignment myfile.apk

the output:


[ERROR][obfuscapk.tool.Jarsigner][resign()] Error during the removal of the old signature: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk'
[ERROR][obfuscapk.obfuscation][sign_obfuscated_apk()] Error during apk signing: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk'
[ERROR][yapsy_loaded_plugin_NewSignature_1.new_signature.NewSignature][obfuscate()] Error during execution of "NewSignature" obfuscator: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk'
[CRITICAL][obfuscapk.main][perform_obfuscation()] Error during obfuscation: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk'

`Traceback (most recent call last):
  File ". . .Obfuscapk\src\obfuscapk\main.py", line 119, in perform_obfuscation
    (obfuscator_name_to_function[obfuscator_name])(obfuscation)
  File ". . .Obfuscapk\src\obfuscapk\obfuscators\new_signature\new_signature.py", line 20, in obfuscate
    obfuscation_info.sign_obfuscated_apk()
  File ". . .Obfuscapk\src\obfuscapk\obfuscation.py", line 507, in sign_obfuscated_apk
    jarsigner.resign(
  File ". . .Obfuscapk\src\obfuscapk\tool.py", line 254, in resign   with zipfile.ZipFile(apk_path, "r") as current_apk:
  File ". . .\Python38-32\lib\zipfile.py", line 1251, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myfile_obfuscated.apk'

Traceback (most recent call last):
  File ". . .\Python38-32\lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File ". . .\Python38-32\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File ". . .Obfuscapk\src\obfuscapk\cli.py", line 133, in <module>
    main()
  File ". . .Obfuscapk\src\obfuscapk\cli.py", line 121, in main
    perform_obfuscation(
  File ". . .Obfuscapk\src\obfuscapk\main.py", line 119, in perform_obfuscation
    (obfuscator_name_to_function[obfuscator_name])(obfuscation)
  File ". . .Obfuscapk\src\obfuscapk\obfuscators\new_signature\new_signature.py", line 20, in obfuscate
    obfuscation_info.sign_obfuscated_apk()
  File ". . .Obfuscapk\src\obfuscapk\obfuscation.py", line 507, in sign_obfuscated_apk
    jarsigner.resign(
  File ". . .Obfuscapk\src\obfuscapk\tool.py", line 254, in resign
    with zipfile.ZipFile(apk_path, "r") as current_apk:
  File ". . .\Python38-32\lib\zipfile.py", line 1251, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: obfuscation_working_dir\\myfile_obfuscated.apk

Here is a very basic APK created with Android Studio , that's still having same issue, 2.6M size
 https://drive.google.com/file/d/15BfjNBM4cfDxncM0dN3UI-LUyPVKWN5W/view?usp=sharing

PS. even using just a Reorder module still pushing Java CPU Utilization to 80%, briefly
skreenr commented 3 years ago

I now see that trying running with minimal configuration per TROUBLESHOOTING fails with the same error No such file or directory: 'obfuscation_working_dir\\TestAstu_obfuscated.apk' , even when using -i option

Is this the anti-repackaging technique thing that arises when using Android Studio, and it's apktool where the journey stops?

skreenr commented 3 years ago

It is apktool issue and it needs to be run with -aapt2 option Thetool.pyneeds to be adjusted by adding --use-aapt2 parameter on apktool build line

ClaudiuGeorgiu commented 3 years ago

Hi @skreenr, the latest version of Obfuscapk already has --use-aapt2 flag that can be passed as argument during the obfuscation, there is no need to modify any file. However, I'm glad you found a solution to the issue đź‘Ť

skreenr commented 3 years ago

Actually I am not in the clear yet:

It worked when I do the "minimal configuration" Resign/Rebuild/NewAlignment thing, If I try adding some Obfuscators, it works OK on Reorder, yet gives same FileNotFoundError: [Errno 2] No such file or directory on FieldRename, CallIndirection. In the log files I don't see exact failure point in obfuscators, all it reports is NewSignature failed with FileNotFound error. Struggling to make sense of it, here is log:

Traceback (most recent call last): File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\main.py", line 141, in perform_obfuscation (obfuscator_name_to_function[obfuscator_name])(obfuscation) File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\obfuscators\new_signature\new_signature.py", line 20, in obfuscate obfuscation_info.sign_obfuscated_apk() File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\obfuscation.py", line 507, in sign_obfuscated_apk jarsigner.resign( File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\tool.py", line 257, in resign with zipfile.ZipFile(apk_path, "r") as current_apk: File "C:\Users\dev\AppData\Local\Programs\Python\Python38-32\lib\zipfile.py", line 1251, in __init__ self.fp = io.open(file, filemode) FileNotFoundError: [Errno 2] No such file or directory: 'obfuscation_working_dir\\xdaapk_obfuscated.apk'

skreenr commented 3 years ago

What I am trying to say, the logging seems to be missing actual failure points in obfuscator modules. We are not seeing "real" errors. How come?

ClaudiuGeorgiu commented 3 years ago

What I am trying to say, the logging seems to be missing actual failure points in obfuscator modules. We are not seeing "real" errors. How come?

Have you tried enabling verbose logs? Do you get additional info about the error?

skreenr commented 3 years ago

I tried earlier and get a flood of msgs, like 1.5M log files Thinking if it's an Error, it should appear in ERROR log

skreenr commented 3 years ago

If you have a tip on what line to look for in Verbose log, I can try

ClaudiuGeorgiu commented 3 years ago

This works for me without errors on Windows with the latest version of Obfuscapk and the apk you provided:

python3 -m obfuscapk.cli --use-aapt2 -o CallIndirection -o FieldRename -o Rebuild -o NewAlignment -o NewSignature C:\Users\me\Desktop\TestAStu.apk
skreenr commented 3 years ago

OK it works with this minimal file. However when I try to run the same command against my production APK I get same Error. I changed log level to ERROR and here is what I see, not really helpful:

ERROR:obfuscapk.tool.Jarsigner:Error during the removal of the old signature: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapk_obfuscated.apk'
ERROR:obfuscapk.obfuscation:Error during apk signing: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapk_obfuscated.apk'
ERROR:yapsy_loaded_plugin_NewSignature_1.new_signature.NewSignature:Error during execution of "NewSignature" obfuscator: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapk_obfuscated.apk'
CRITICAL:obfuscapk.main:Error during obfuscation: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapk_obfuscated.apk'
Traceback (most recent call last):
  File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\main.py", line 141, in perform_obfuscation
    (obfuscator_name_to_function[obfuscator_name])(obfuscation)
  File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\obfuscators\new_signature\new_signature.py", line 20, in obfuscate
    obfuscation_info.sign_obfuscated_apk()
  File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\obfuscation.py", line 507, in sign_obfuscated_apk
    jarsigner.resign(
  File "C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\tool.py", line 257, in resign
    with zipfile.ZipFile(apk_path, "r") as current_apk:
  File "C:\Users\dev\AppData\Local\Programs\Python\Python38-32\lib\zipfile.py", line 1251, in __init__
    self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapk_obfuscated.apk'

DEBUG-level log is 1.8M size with a lot of repeating lines

skreenr commented 3 years ago

I modified logging code so now it logs obfuscator name (first 5 lines below) and what I see is every obfuscator fires the same Error:

ERROR:root:obfuscator_name:  CallIndirection
ERROR:root:obfuscator_name:  FieldRename
ERROR:root:obfuscator_name:  Rebuild
ERROR:root:obfuscator_name:  NewSignature
ERROR:obfuscapk.tool.Jarsigner:Error during the removal of the old signature: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapp_obfuscated.apk'
ERROR:obfuscapk.obfuscation:Error during apk signing: [Errno 2] No such file or directory: 'obfuscation_working_dir\\myapp_obfuscated.apk'
ERROR:yapsy_loaded_plugin_NewSignature_1.new_signature.NewSignature:Error during execution of "NewSignature" obfuscator: [Errno 2] No such file or directory: 'obfuscation_working_dir\\xda_obfuscated.apk'
CRITICAL:obfuscapk.main:Error during obfuscation: [Errno 2] No such file or directory: 'obfuscation_working_dir\\xda_obfuscated.apk'

Is it perhaps absolute path vs relative path thing? like it's getting relative path instead of absolute and can't see a file?

ClaudiuGeorgiu commented 3 years ago

Please try using the latest version of Obfuscapk, now we use Apksigner instead of Jarsigner, this might help.

skreenr commented 3 years ago

My hunch is it still can be Apktool related, in that when I manually decompile then rebuild APK with APK tool, the production APK with FCM shows one extra dir in Apktool structure named 'unknown' alongside original, res, smali etc. Inside that dir I see FCM related files, "properties" files

skreenr commented 3 years ago

Examining DEBUG-level log, here is a block of interest, which repeats several times:

``DEBUG:yapsy:correct subclass tests failed for: builtins in C:\Users\dev\Documents\DECOMPILE\Obfuscapk\src\obfuscapk\obfuscators\advanced_reflection Traceback (most recent call last): File "C:\Users\dev\AppData\Local\Programs\Python\Python38-32\lib\site-packages\yapsy\PluginManager.py", line 535, in loadPlugins is_correct_subclass = issubclass(element, self.categories_interfaces[category_name]) File "C:\Users\dev\AppData\Local\Programs\Python\Python38-32\lib\abc.py", line 102, in subclasscheck return _abc_subclasscheck(cls, subclass) TypeError: issubclass() arg 1 must be a class

skreenr commented 3 years ago

I installed the current release ZIP, have built the production APK with all obfuscators – the output file is 2M bigger in size! interesting Why – yet it fails to install on the phone. Probably issue is with a signature. What sigs are you guys using there?

Tried passing key-passwd and -alias on command line, now getting Python error:

PermissionError (WinError 31) A device attached to the system is not functioning Found this article https://stackoverflow.com/questions/50085366/permissionerror-winerror-31-a-device-attached-to-the-system-is-not-functioning which advises workaround chcp 1252
yet it won't work

skreenr commented 3 years ago

I was able to rebuild obfuscated APK, without correct APKSigner sig, after closing Python window and reopening it for new session. Did use--ignore-libsoption, and obfuscators complete the run reasonably fast.

Here is just the beginning a part of the Logcat related to the Error. It shows multiple Firebase references: full segment is at https://drive.google.com/file/d/1GZk0hlnC4KYdUlOoQwQxIXBbw3g_FvXm/view?usp=sharing


`2021-08-12 19:10:25.493 7329-7329/? E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.mydomain.myapp, PID: 7329
    java.lang.NoSuchFieldError: No static field LEGACY_INSTANCE of type Lcom/google/android/datatransport/cct/CCTDestination; in class Lcom/google/android/datatransport/cct/CCTDestination; or its superclasses 

(declaration of 'com.google.android.datatransport.cct.CCTDestination' appears in /data/app/com.mydomain.myapp-eZAt2abbBEiwKs_y8gXKSQ==/base.apk)
        at com.google.firebase.datatransport.TransportRegistrar.lambda$getComponents$0(com.google.firebase:firebase-datatransport@@17.0.3:38)
        at com.google.firebase.datatransport.TransportRegistrar$$Lambda$1.create(Unknown Source:0)
`

Can you explain how APKSign needs to be called in this new version with a keystore?, like examples please Had to run APKSign manually, the output APK verified and installed, however it won't run, just exits right away

Any tips appreciated, like How to isolate the error (is it signing or elsewhere?)

skreenr commented 3 years ago

That minimal APK built with AStudio I shared earlier – here I decided to test theory that Google libs like Firebase break Obfuscapk. What I did was, added Firebase functionality to the project and now, after obfuscating and resigning new APK, I get the same kind of behavior on device, in that the app abruptly exits at launch w/o showing anything. Just like my production APK. Here is a link to APK as it comes from AStudio: https://drive.google.com/file/d/1jhaoySLkf6tOmK2dE1RY9_f3a4rTc5Ze/view?usp=sharing

ClaudiuGeorgiu commented 3 years ago

after obfuscating and resigning new APK, I get the same kind of behavior on device, in that the app abruptly exits at launch w/o showing anything

Did you use all the available obfuscators? Using --ignore-libs should have solved the issue, but since you already tried that, there must be at least one obfuscator that doesn't play well with Firebase. You can try finding the problematic obfuscator(s) as described here, then exclude from the list those that break the app when using Firebase.

skreenr commented 3 years ago

I should have mentioned I do use --ignore-libs Looking at the directory structure creaed by APKTOOL I see one new directory added "unknown" , filled with multiple Firebase PROPERIES files 1K size each

skreenr commented 3 years ago

Claudiu, on your msg: using 3 obfuscators in a chain: I do need them all to achieve protection! The thing is, without Firebase in the mix both TestAStu and the bigger production APK successfully obfuscate, install and run on a device. Adding Firebase leads to the app's immediate exit on start-up with error in the log hinting Failure to initialize Firebase: com.google.firebase.provider.FirebaseInitProvider.onCreate() , see https://drive.google.com/file/d/1GZk0hlnC4KYdUlOoQwQxIXBbw3g_FvXm/view

What I did next was, I wanted to check an assumption the failure is with APKTool incorrectly handling Firebase– by decompiling TestASTwFirebase with APKTool, then rebuilding and resigning it. I expected it will fail to run, I was wrong: the app installs on a device, runs and receives Firebase notifications just like original APK does.

Hmm, I have no ideas at the moment. All I can think of is, if we can find way to import non-Firebased obfuscated classes in AStudio to add Firebase functionality to a final build..

skreenr commented 3 years ago

Problematic obfuscator turns out to be FieldRename When I use just it with Rebuild/Align/Sign I get the same app crash on start-up as reported above. Other two obfuscators I use, each of them produced a working APK when used alone.

To just drop FieldRename from the list of 3 I am using would weaken obfuscation, that's the concern I would much rather find how to fix it, get it behave like others do

skreenr commented 3 years ago

Solved this: after Googling for exact error msg, found some SOF posts about how it's important that various Firebase libraries versions listed in build gradle version-match. After changing version # some, it worked flawlessly. The curious thing of course is, Why the error – the app abruptly quits at startup– only happens after Obfuscapk, and not at all with the original build.

skreenr commented 3 years ago

Closing this