search

MuntashirAkon / AppManager

A full-featured package manager and viewer for Android
https://muntashirakon.github.io/AppManager/
Other
4.8k stars 270 forks source link

APK Editor (RE-Build APKs and More) #138

Open RickyM7 opened 3 years ago

RickyM7 commented 3 years ago

Describe the feature

Basically an option to edit each part (manifest, files, data, etc.) of an app or apk/apks file, etc.

Describe the solution you'd like

It would be great if this were similar to what is seen in the APK Editor PRO app. This app has several useful features.

Describe alternatives you've considered

It would be great if this were divided into three options:

  1. Common Edit
  2. Full Edit
  3. Edit Data

All these options would be the same as what is seen in the APK Editor PRO. (See the "Additional context" section to learn what these options look like)

Related issues

6, #14, #36, #116, #165, #176, #141 and #222 (There are probably more issues but I have not found them).

Additional context

APK Editor PRO features

Feature Description Example 1 Example 2
XML File Edit Basically the xml files of the apk can be edited (This is included in Full Edit mode) IMG_20201101_101222 APKEditor XML
Common Edit Basic parts of AndroidManifest.xml can be edited (This is included in Full Edit mode but still worth having a separate tab for this) APKEditor Common Edit
Simple Edit Just an option to replace the apk files (This is included in Full Edit mode) APKEditor Simple Edit
Full Edit (Resource RE-Build) In this option everything can be changed, from AndroidManifest.xml to .smali files. It is possible to edit the file codes, replace them, delete them, edit permissions, etc. Full Edit home 1 Full Editor Home Full Edit home 2 IMG_20201101_100327
Manifest Editor Anything from Manifest can be changed here (A piece of Full Edit mode) Manifest Editor
Permissions Editor This is in the manifest editor tab (A piece of Full Edit mode) Permitions Editor IMG_20201101_100640
Strings Editor Strings of any language can be edited and other strings can be added (A piece of Full Edit mode) Strings Editor
Smali Editor Edit .smali files (A piece of Full Edit mode) Smali Editor
Edit Data (Root) Edit database, preferences, etc. of already installed apps editdata

Developement Info

Features (In the order of implementation):

Tools (There will be some one-click special tools):

Future Plans (May never see daylight!):

Atrate commented 3 years ago

Mostly, if not completely covered by #6 but this app's UI paradigms could really come in handy

MuntashirAkon commented 3 years ago

I think it's completely covered by #6. With the introduction of bundled apps, thinks are a bit complicated though (such as all splits have to be signed with the same signing key, etc.). Adding support for APK editing is a quite complicated task and it would be better if somebody listed the features supported by the APK Editor.

One other complication is that the open source APK editors such ApkTool, jadx, are not completely free software as they use prebuilt jar files (APK Editor, etc. are using these projects, therefore, they also have it) without any build instructions. I have no idea how I can compile Android.jar file (I never really worked on Android ROMs). I want to make sure that AM always remains a free (as in libre) software so that it won't die away when I'm not around here to continue its development, and this significantly hinders my efforts. If anybody can figure out how to build Android.jar file, then and only then I can proceed.

In the mean time, I will add support for signing APK files.

RickyM7 commented 3 years ago

@MuntashirAkon Got it, I also think it's okay if these features aren't added or worked only in the future, mainly because of the complexity that would be working on it. Besides, as the issue #6 cover this issue, I'll close it now. Thanks for the reply and thanks for your work.

MuntashirAkon commented 3 years ago

6 is already closed since it contained too many off-topic comments.

Cyberavater commented 3 years ago

Yes, it'd be nice if AM could repack .apks/.apkm/.xapk/.zip(.apks) to a universal .apk.

MuntashirAkon commented 3 years ago

Yes, it'd be nice if AM could repack .apks/.apkm/.xapk/.zip(.apks) to a universal .apk.

It is possible using the bundletool. But we need a more cleaner solution. Because if this is going this way, the app size of AM would be around 50 MB when installed.

Cyberavater commented 3 years ago

Yes, it'd be nice if AM could repack .apks/.apkm/.xapk/.zip(.apks) to a universal .apk.

It is possible using the bundletool. But we need a more cleaner solution. Because if this is going this way, the app size of AM would be around 50 MB when installed.

Why size would matter? An app that can do anything related to an app would surely have some weight.

MuntashirAkon commented 3 years ago

Why size would matter? An app that can do anything related to an app would surely have some weight.

It matters because not all people are going to use this feature, including myself (why would I need an apk converter when bundled app works on my phone?). Even if somebody uses the feature, it's very unlikely that s/he's going to use it everyday. Also, you've have to consider the people who do not have enough storage (like myself). Cost-benefit relationship is an important issue when it comes to developing an app.

Cyberavater commented 3 years ago

Why size would matter? An app that can do anything related to an app would surely have some weight.

It matters because not all people are going to use this feature, including myself (why would I need an apk converter when bundled app works on my phone?). Even if somebody uses the feature, it's very unlikely that s/he's going to use it everyday. Also, you've have to consider the people who do not have enough storage (like myself). Cost-benefit relationship is an important issue when it comes to developing an app.

Hmm, then how about having a separate app as APK editor? I know there already exists one but it's missing many features.

MuntashirAkon commented 3 years ago

Hmm, then how about having a separate app as APK editor? I know there already exists one but it's missing many features.

No, that would introduce the Xy problem. It's better to solve the real problem instead.

APK editor is not currently possible because of the reason I've stated above. Android.jar, aapt, aapt2 are the three requirements for the APK editor and unfortunately, I don't know how to build Android.jar.

MuntashirAkon commented 3 years ago

It looks like I've found a way to compile android.jar.

MuntashirAkon commented 3 years ago

Features (in the order of implementation)

  1. Signing apk. For edited apks, signing with a key is mandatory. But many people wants to sign existing apks as well (there will be an option in settings for that). AM will allow user to use their own signing keys instead of just the test keys.
  2. Explore apk.
  3. Decompile apk. An option to decompress the resources and the dex files and save them in a specified location. There will be a meta file along with the files which can be used to recompile the project again.
  4. Manifest editing. Option to edit the app manifest.
  5. Recompile apk. After editing the files with the editor of your choice, you can recompile the project by importing it.
  6. Smali editor. A text editor will be added to allow users to edit the smali files.
  7. Resource editor. A slightly advanced editor to edit resource files.

Tools There will be some one-click special tools:

  1. App cloning. Changing package name in one click. Unfortunately, this will not work for apps that doesn't use unique names for authority.
  2. Remove tracker components. Removing the tracking signature from the App Manifest. This may result the app to crash.
  3. Remove app permissions. Since normal permissions can't be removed, this is the only way to remove these permissions.
MuntashirAkon commented 3 years ago

Support for patches This is just a future plan, it may never see daylight!

Apart from the patch schema developed by the author of APK Editor, AM can store patches for the apk files (using version control system such as git) to allow users to apply them again in future. This could be helpful (but not always reliable) for the modified apps if they need an update, in which case AM will decompile the apps, apply the previously saved patches, compile and sign them before displaying the install screen. This way the user doesn't have to skip updates.

Atrate commented 3 years ago

Apart from the patch schema developed by the author of APK Editor

In case this gets implemented, support for Lucky Patcher's patch schema would be beneficial since LP already has tons of patches for popular apps

MuntashirAkon commented 3 years ago

In case this gets implemented, support for Lucky Patcher's patch schema would be beneficial since LP already has tons of patches for popular apps

Do they have any patching guide that I can follow (like APK Editor does)?

ghost commented 3 years ago

Apart from the patch schema developed by the author of APK Editor

In case this gets implemented, support for Lucky Patcher's patch schema would be beneficial since LP already has tons of patches for popular apps

And most of those patches are used to crack apps. Lucky patcher has bad reputation among app devs because of the same reason.. The day won't be far if AM implement something like that and get flagged by Play Store Protection as malicious.

Cyberavater commented 3 years ago

Any app related to high-level apk editing is marked as malicious by Play Store. Even App Cloner, (the app they made a fortune from) was banned from GP. But I do agree, most Dev would blacklist AM too if it starts that kind of high-level apk editing. Heck, they even blacklist Adblockers.

MuntashirAkon commented 3 years ago

And most of those patches are used to crack apps. Lucky patcher has bad reputation among app devs because of the same reason.

I agree. AM isn't a software for cracking apps. But it will definitely provide features like APK editing because it gives users freedom as well as privacy. Patch schema used by APK Editor is just a shorthand for lengthy, boring tasks. So, it won't be a problem. I don't know what the state of LP is now (I haven't used it in years) but there are valid reasons for it to be flagged as malicious:

  1. It's sole purpose is hacking/cracking an app
  2. It's a closed source app with extensive obfuscation (you cannot understand what it does simply by decompiling it)
  3. It provides run-time patching for apps instead of modifying the APK itself which is a major violation aka "cracking" since the APK itself is signed by the author of the app.

AM will only modify the APK file which has to be signed by either a public/generic key or by a custom key. It means it would respect the signature verification methods if the author of the app implemented such features with sufficient obfuscation (using Proguard or Dexguard) so that a person with malicious intention won't be able to modify it. The app can also supply a EULA to forbid the users from modifying the APK in which case if the app is modified and distributed, the app authors can take legal actions against the user but AM won't be responsible for it because by using AM the user has already consented that neither AM nor I, the developer, will take any responsibility for the user's action.

Even App Cloner, (the app they made a fortune from) was banned from GP.

App Cloner violated GP policy which is why it got deleted from the store. It's not marked as malicious app. APK Editor, which also violated GP is in the play store because it complied with the policy by removing the features that allowed changing app contents.

MuntashirAkon commented 3 years ago

Signing APK files before installing is added via a6a21a03

RickyM7 commented 3 years ago

@MuntashirAkon I just realized that I did not quote these features of apk editor in this issue: Other Options And within these options is the option to edit the signature key: Custom Key Setting I think this is a feature that can be very useful for many people.

MuntashirAkon commented 3 years ago

@RickyM7 Read the point no. 1 again. It reads like this:

But many people wants to sign existing apks as well (there will be an option in settings for that). AM will allow user to use their own signing keys instead of just the test keys.

RickyM7 commented 3 years ago

Read the point no. 1 again. It reads like this:

@MuntashirAkon Of course, I'm sorry, I just edited this part and I didn't remember. Anyway, there is a simple design example that you can use for this feature if you want.

MuntashirAkon commented 3 years ago

Anyway, there is a simple design example that you can use for this function if you want.

AM will have more options than this including the most widely used Java KeyStore (JKS). AM will also support creation of signing keys rather than just importing them.

RickyM7 commented 3 years ago

AM will have more options than this including the most widely used Java KeyStore (JKS). AM will also support creation of signing keys rather than just importing them.

That's really great. Changing a little the subject, I think the AM crashed on my phone because of the new apk signature feature. Can I send the crash log here or should I create a new issue?

MuntashirAkon commented 3 years ago

Can I send the crash log here or should I create a new issue?

Create a new issue.

RickyM7 commented 3 years ago

@MuntashirAkon I added the "Edit Data" feature in Additional context (last feature of the list). This feature is for rooted users only and so I ended up forgetting to add it before.

MuntashirAkon commented 3 years ago

I added the "Edit Data" feature in Additional context (last feature of the list). This feature is for rooted users only and so I ended up forgetting to add it before.

Possible duplicate of #14, shared preferences can already be edited.

MuntashirAkon commented 3 years ago

It looks like I'm going to implement 5 before 4. So, 5 (recompile) should come before 4 (manifest editing).

gnuhead-chieb commented 3 years ago

APK Explorer & Editor gets supporting dex decompilation(to smali) https://github.com/apk-editor/APK-Explorer-Editor/commit/cfff6474463cfbf632f335a9c99ac67737378c83

MuntashirAkon commented 3 years ago

APK Explorer & Editor gets supporting dex decompilation(to smali) apk-editor/APK-Explorer-Editor@cfff647

It's not difficult to implement a feature, planning is. Incorporating a new feature to the existing ones is the most difficult task. That's why the steps were chosen very carefully and will be proceeded that way.

That being said, there's already an open source (but GPL-2.0 licensed) app available called Apktool-Android which has many features, but for some reason, the developer stopped working on it.

MuntashirAkon commented 3 years ago

Update: I'm current working on reviving TextWarrior (https://github.com/MuntashirAkon/TextWarrior), the mother of (almost) all code editors on Android. The aim of the project is to create basic code editor with special support for Smali to Java (latter is read-only) conversion. The project will be integrated to App Manager once it's completed (considering the amount of efforts and learning it requires, it won't be completed anytime soon).

In the other news, I've already integrated a basic file manager which will be further enhanced with basic functionalities as well as support for exploring APK files.

Possible designs for basic APK editing Basic APK editing involves modification of the package name, label, version (code), SDK, and the manifest file. All of them are currently located in the app info tab. So, instead of creating a separate UI for them, users should be able to edit them by activating the edit mode i.e. when edit mode is enabled in settings, an edit button will be displayed at the bottom right side of the tab (and clicking on the button will activate edit mode). When edit mode is activated, users will be able to edit any supported options by simply clicking on them. For manifest editing, when the manifest tab is loaded (which will also have edit button) and the edit button is pressed, the edit mode will be activated. (Since app info page and the manifest file offers the same features, they will be separated.)

When the edit mode is on, APK explorer will have to option to replace contents of the APK file.

Any comments are welcome.

MuntashirAkon commented 3 years ago

Update 2: It's very difficult to work on anything these days. I already have to maintain at least six external libraries for App Manager, mostly because they wouldn't support Android, or even if they support Android, they wouldn't support API 21-25. In this ever growing list of libraries, the latest addition is going to be Jadx whose developers have thus far refused to make its core library compatible with Android (we can still use it but due to its extensive use of Java 1.8 or later syntaxes, it's unusable with d8 (API limitations)). I could use old version of Jadx (like what others do) but that would not be something I would do (as it would introduce xy problem). However, if you're paying attention, you will know that I've already integrated Smali/Baksmali and Jadx (later isn't working with universal APK option due to compatibility issues) in d86d3ba1 and bec0b3e0. While they're currently used for scanning the APK files, they can also be used for decompiling APK files. I've decided to postpone the development of TextWarrior for now as we can use external editors to do its job, and focus on more important issues such as exploring an APK file (2) and decompiling its DEX/resources files (3), and none can do better at decompiling resources file than Jadx.

At the same time, I am also trying to improve the scanner page (#133) which also seems to be problematic due to Quark Engine and APKiD being written in Python. As a result, I am also working on a minimal Java version of Quark Engine, and later I will figure out a way to convert Yara rules in APKiD so that we can analyse APK files without needing to take the trouble of creating a JNI wrapper for Yara. If I am able to succeed in these two features, the visions that I have will be completed.

I won't be available for several days. So, I thought I should share another update in case someone is interested in improving these features. Thanks.

MuntashirAkon commented 2 years ago

APK Explorer is added in 054c1071. This is currently a bare-bones implementation with browsing capabilities only.

MuntashirAkon commented 2 years ago

it looks like Smali/Baksmali has Java compatibility issues too! The scanner isn't working in Android < Oreo.

MuntashirAkon commented 2 years ago

After bbb9859d, the XML output should be identical to the ones by APK Editor (XML file edit mode), except that the whitespaces are different.

MuntashirAkon commented 2 years ago

5da4ab6c Marks the end of APK Explorer. We will now focus on basic functionalities available on APK Editor such as Extract, Replace, Delete and Rename. Our explorer has already equipped with everything necessary to proceed to the next level. I am also thinking of adding a DEX, APKM, APKS and XAPK explorer (we've already cleverly named it only Explore instead of Explore APK or something similar) but the latter three would make things a bit complicated for me.

For now, we will not actually decompile anything. Both APK files and DEX files are explored using a custom-made virtual file system, meaning while the user can browse both the APK and the DEX files, they aren't actually saved anywhere (which is good as the access time is very fast). In the recent future, when a user modifies a file, the changed file will be saved to disk as usual, and when they finally chose to repack the APK, all files will be extracted, compiled (when necessary) and repacked so that the user have to sit and wait only during this period. Also notice that, we haven't yet touched the resources.arsc file. This is because in order to recompile resources, we would need AAPT2 which has an android.jar dependency (which I can provide as an asset). But the issue is AAPT is very big in size and we actually do not need all functions of it. I've already created a working version of zip-align library and the apksig library has been integrated a long time ago. So, supporting resources.arsc would take some time. However, the future of resource editor has already been threatened by obfuscators like AndResGuard.

Gradually, after adding a text editor to actually edit XML and Smali files, I will try to improve the search engine. APK Editor excels at searching and so is my target.

gnuhead-chieb commented 2 years ago

probably this apktool will work with android https://github.com/Mobile-IoT-Security-Lab/HideDroid/tree/main/apktool

it bundled in this app https://apt.izzysoft.de/fdroid/index/apk/it.unige.hidedroid

MuntashirAkon commented 2 years ago

probably this apktool will work with android

This app can't upgrade SDK version because of its reliance on Apktool (W^X issue), and Apktool itself relies on prebuilt version of AAPT and Android.jar. Also, there's no existing proof that it works below Android Oreo (and they don't support Android Lollipop either). So, my guess is it's the same as the original with a few modification to suite their needs. Libraries like Apktool (we don't need) and JADX (both of which were never intended for use in Android or as libraries in general) are highly bloated software (use of jCommander, SLF4J logger, etc.) and need to be debloated, and Smali/baksmali relies on certain libraries which can be replaced with Android frameworks or AndroidX libraries. Currently, the resultant APK file after including smali/baksamli and JADX is too large. This might be improved by writing a few proguard rules but the resultant APK would still be very large.

MuntashirAkon commented 2 years ago

JADX currently has 544 incompatibilities with Android N or earlier which needs to be fixed.

Screenshot 2021-10-21 at 13 21 50

kryptobolt07 commented 2 years ago

Well you can ask the dev of apktoolm for help ..he is a really nice guy https://github.com/Maximoff https://4pda.to/forum/index.php?showuser=4424665

MuntashirAkon commented 2 years ago

Well you can ask the dev of apktoolm for help ..he is a really nice guy

Well, I have an academic background and I take plagiarism very seriously, for example, APKToolM author's copying a GPL'd code and making it closed source is a serious violation and a crime.

kryptobolt07 commented 2 years ago

Well you can ask the dev of apktoolm for help ..he is a really nice guy

Well, I have an academic background and I take plagiarism very seriously, for example, APKToolM author's copying a GPL'd code and making it closed source is a serious violation and a crime.

didn't know that ...but hey the dev of apktoolm is a russian and in russia basically intellectual property(or licences) is(are) given no importance...(You are correct in what you said not .. I don't mean to argue about what you said..++you are really a great and ethical dev)

MuntashirAkon commented 2 years ago

It appears Jadx library has too many Java 8 dependencies (it was ensured that Jadx only has Java 8 dependencies and not later) that cannot be easily ported back to Android Nougat or earlier. At this point, it might be better to completely separate Jadx to create an android-specific library since we do not actually need most of its feature.

MuntashirAkon commented 1 year ago

I shall add most of these features in v4.0.0 (mostly skipping the resource/strings editor). For the rests, separate issues need to be created as this issue is too big to keep track of.

KIDU333 commented 1 year ago

Help i wont to creat a zip patch file for apk editor pro what tools do i need

MuntashirAkon commented 1 year ago

It appears Jadx library has too many Java 8 dependencies (it was ensured that Jadx only has Java 8 dependencies and not later) that cannot be easily ported back to Android Nougat or earlier. At this point, it might be better to completely separate Jadx to create an android-specific library since we do not actually need most of its feature.

Fixed.

MuntashirAkon commented 1 year ago

Sora Editor has now been integrated. This still needs some improvements.

With the help of ARSCLib, we can further accelerate the process. Currently, more testing is needed for the virtual file system, and it is also necessary to fix a plan for how a project shall be handled. My initial thinking is as follows:

File format Could be a folder or a [compressed] blob. Folder could be preferred because during compilation, all files may have to be extracted anyway. Other similar projects such as ApkTool uses a folder.

File contents

  1. A metadata file to store basic metadata
  2. A map file to keep track of the changed files since we may not be extracting all the files at first
  3. The original APK file
  4. Extracted contents

Rebuilding APK files

  1. Reversing is really about trying different combinations to find out the correct/best ones. Hence, recompilation is frequently required. For resources (.arsc) and DEX (.dex) files, detect changes before compiling them in order to improve the subsequent build time.
  2. Similar to other IDE, an option to build and run will be provided which will (if possible) automatically install and run the app after building it
  3. For apps supporting multidex, it might also be possible to pack frequently edited classes in a separate dex. But this requires a feasibility study.

Support/follow ApkTool format? I haven't found any specification for its format. So, I don't really know if it's possible. If anybody's familiar with it, please let me know.

ElsAr4e commented 10 months ago

Just wanted to write a feature request:

Remove tracker components:

If you extract the apk please allow to remove the trackers.