search

iBotPeaches / Apktool

A tool for reverse engineering Android apk files
https://apktool.org/
Apache License 2.0
19.88k stars 3.56k forks source link

[BUG] Invalid drawable declarations generated when decompiling Instagram 207.0.0.39.120 #2670

Open grishka opened 2 years ago

grishka commented 2 years ago

Information

  1. Apktool Version (apktool -version) - 2.6.0
  2. Operating System (Mac, Linux, Windows) - Mac
  3. APK From? (Playstore, ROM, Other) - Play Store

Facebook uses some unusual build system for their apps. Resources end up packaged inside the apk in a single folder named r, with some files referenced by more than one entry inside resources.arsc. While apktool does undo most of it correctly, it generates a file res/values/drawables.xml like this:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <item type="drawable" name="chalk_brush">R||AQAEH1e24i3A|512|512|160</item>
    <item type="drawable" name="circle">R||AQAAoKYEg-_S|256|255|160</item>
    <item type="drawable" name="direct_dark_mode_inbox_reply_pill_background_gray">@drawable/direct_inbox_reply_pill_background_gray_night</item>
    <!-- more proper resource aliases -->
    <item type="drawable" name="direct_thread_gifs_place_holder_item_shhmode">@drawable/direct_thread_gifs_place_holder_item_night</item>
    <item type="drawable" name="fr_frame">R||AQAAkUF_KkOv|1024|1024|160</item>
    <item type="drawable" name="fr_lut">R||AQAAyPlUeUNj|1024|32|160</item>
    <item type="drawable" name="frbw_frame">R||AQAAk2w7mYkq|1024|1024|160</item>
    <item type="drawable" name="frbw_lut">R||AQAB6iBI8j8n|1024|32|160</item>
    <item type="drawable" name="frbw_overlay">R||AQACuRw-mWCq|1024|1024|160</item>
    <item type="drawable" name="in_call_notif_tv_icon">R||AQAGKjMnng39|120|122|160</item>
    <item type="drawable" name="inline_searchbox_background_shhmode">@drawable/inline_search_box_background_night</item>
    <item type="drawable" name="input_background_shhmode">@drawable/input_background_night</item>
    <item type="drawable" name="instagram_add_filled_24">I1800B7</item>
    <item type="drawable" name="instagram_add_outline_24">I18007B</item>
    <!-- a whole lot more of these nonsensical lines -->
    <item type="drawable" name="ll_font">R||AQACveI2bHlA|160|96|160</item>
    <item type="drawable" name="ll_leaks">R||AQABjmVlm-FX|1024|1024|160</item>
    <item type="drawable" name="ll_lut">R||AQADDu-9mCiw|1024|32|160</item>
    <item type="drawable" name="ll_noise">R||AQADKNuSWsE5|1024|1024|160</item>
    <item type="drawable" name="marker">R||AQABJEwQfEtS|256|256|160</item>
    <item type="drawable" name="media_additional_reactors_circle_shhmode">@drawable/media_additional_reactors_circle_night</item>
    <item type="drawable" name="meme_emphasis">R||AQACINpbVTEL|1301|371|160</item>
    <item type="drawable" name="neon_bulb_asset">R||AQABxh689A9n|256|256|160</item>
    <item type="drawable" name="ps_lut">R||AQABHygJLIb3|1024|32|160</item>
    <item type="drawable" name="radial_asset">R||AQADhGZJhKUF|256|256|160</item>
    <item type="drawable" name="rainbow_brush">R||AQAAeJIP-Nxa|256|256|160</item>
    <item type="drawable" name="textedit_background_error_shhmode">@drawable/textedit_background_error_night</item>
    <item type="drawable" name="textedit_background_shhmode">@drawable/textedit_background_night</item>
    <item type="drawable" name="unified_inbox_message_mask_no_border_shhmode">@drawable/unified_inbox_message_mask_no_border_night</item>
    <item type="drawable" name="unified_inbox_message_mask_shhmode">@drawable/unified_inbox_message_mask_night</item>
    <item type="drawable" name="unified_inbox_my_message_mask_shhmode">@drawable/unified_inbox_my_message_mask_night</item>
    <item type="drawable" name="vhs_glitch_0">R||AQADL3Xyljfu|1024|1024|160</item>
    <item type="drawable" name="vhs_glitch_1">R||AQABaRWCTW4_|1024|1024|160</item>
    <item type="drawable" name="vhs_lut">R||AQABd-1upOoH|1024|128|160</item>
    <item type="drawable" name="APKTOOL_DUMMY_bd" />
    <item type="drawable" name="APKTOOL_DUMMY_105" />
    <item type="drawable" name="APKTOOL_DUMMY_199" />
    <item type="drawable" name="APKTOOL_DUMMY_19a" />
    <item type="drawable" name="APKTOOL_DUMMY_1d4" />
    <item type="drawable" name="APKTOOL_DUMMY_1d5" />
    <!-- and some more of these too — not sure why they're here at all -->
</resources>

Obviously, building this as is results in an app that crashes when it first tries to make use of a resource that decoded incorrectly.

I don't know Android internals nearly well enough, but might it be that there's some newer kind of resource reference in resources.arsc, and these are meant to be @drawable/something references?

Stacktrace/Logcat

10-04 00:33:21.495 22922 22922 D AndroidRuntime: Shutting down VM
10-04 00:33:21.497 22922 22922 E AndroidRuntime: FATAL EXCEPTION: main
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Process: com.instagram.android, PID: 22922
10-04 00:33:21.497 22922 22922 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.instagram.android/com.instagram.mainactivity.MainActivity}: android.view.InflateException: Binary XML file line #34: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3654)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3806)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2267)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.os.Handler.dispatchMessage(Handler.java:107)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.os.Looper.loop(Looper.java:237)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.ActivityThread.main(ActivityThread.java:8167)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at java.lang.reflect.Method.invoke(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.view.InflateException: Binary XML file line #34: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.view.InflateException: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: java.lang.reflect.InvocationTargetException
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at java.lang.reflect.Constructor.newInstance0(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.view.LayoutInflater.createView(LayoutInflater.java:854)
...
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.view.LayoutInflater.inflate(LayoutInflater.java:481)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at X.1Zj.A0P(:23)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at androidx.appcompat.app.AppCompatActivity.setContentView(:268435463)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.instagram.base.activity.BaseFragmentActivity.onCreate(:14)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.instagram.mainactivity.MainActivity.onCreate(:349)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.Activity.performCreate(Activity.java:7963)
...
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.app.ActivityThread.main(ActivityThread.java:8167)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at java.lang.reflect.Method.invoke(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_arrow_left_outline_24 with resource ID #0x7f0804ab
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_arrow_left_pano_outline_24 with resource ID #0x7f0804ae
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: File I180158 from drawable resource ID #0x7f0804ae
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.content.res.ResourcesImpl.loadDrawableForCookie(ResourcesImpl.java:999)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.content.res.ResourcesImpl.loadDrawable(ResourcesImpl.java:735)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.content.res.Resources.getDrawableForDensity(Resources.java:965)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at X.11v.getDrawableForDensity(:65)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.content.res.Resources.getDrawable(Resources.java:880)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at X.1Xt.A00(:33)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at X.1Xu.get(:18)
10-04 00:33:21.497 22922 22922 E AndroidRuntime:    at android.content.res.ResourcesImpl.loadDrawable(ResourcesImpl.java:715)
...
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: Caused by: java.io.FileNotFoundException: I180158
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    at android.content.res.AssetManager.nativeOpenNonAsset(Native Method)
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    at android.content.res.AssetManager.openNonAsset(AssetManager.java:992)
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    at android.content.res.ResourcesImpl.loadDrawableForCookie(ResourcesImpl.java:983)
10-04 00:33:21.498 22922 22922 E AndroidRuntime:    ... 58 more

Steps to Reproduce

  1. apktool d instagram.apk
  2. apktool b instagram

Frameworks

If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files (.apks that live in /system/framework or /system/priv-app)

It is not.

APK

If this APK can be freely shared, please upload/attach a link to it.

https://www.apkmirror.com/apk/instagram/instagram-instagram/instagram-instagram-207-0-0-39-120-release/instagram-207-0-0-39-120-android-apk-download/download/

Questions to ask before submission

  1. Have you tried apktool d, apktool b without changing anything? Yes, that's exactly what I did.
  2. If you are trying to install a modified apk, did you resign it? Yes.
  3. Are you using the latest apktool version? Yes.
grishka commented 2 years ago

Duplicating here from the chat. I worked around this by decompiling with --no-res, extracting the /r directory from the apk into /unknown in the project, and adding all the files under /r to unknownFiles in apktool.yml. This results in a fully working rebuilt apk.

Downside? I can't modify the resources. Not that I intended to, anyway. This would most probably work with other Facebook apps (Facebook itself, Messenger, and probably WhatsApp) as well.

p.s. TIL that Android 11 is really picky about apks and requires 4-byte alignment and a v2 signature if you target it.

MartinDupont commented 2 years ago

The drawable declarations are actually being generated correctly, in the sense that they are being directly copied from the resources.asrc table. For example, for me the install always fails with Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_facebook_circle_filled_24 with resource ID #0x7f08069e

When I look up the generated entry for that drawable in values/drawables, I find an apparently nonsensical entry: <item type="drawable" name="instagram_facebook_circle_filled_24">I18006C</item>

However, that row is actually in the resources.asrc table. It looks like this: ID name default 0x7f08069e instagram_facebook_circle_filled_24 I18006C

I don't know what "I18006C" is actually supposed to mean. But there are some xml entries that are decipherable. For the entries that look like this one:

<item type="drawable" name="theme_halloween_preview_icon_light">R||AQAC7IVDw74H|150|150|240</item>

The AQAC7IVDw74H is actually a base64url encoded string, which instagram parses to the url https://lookaside.facebook.com/ras/v2/?id=AQAC7IVDw74H which downloads a nice picture of a jack-o-lantern. So they clearly have some custom code for downloading resources on the fly.

So, these resources are being decoded by apktool correctly. And you can look at the resources.asrc fille of the recompiled apk and see that they are also being reencoded correctly too.

This begs the question: why is the recompiled program crashing then? I have no idea.

iBotPeaches commented 2 years ago

I appreciate the research you did as I read my email on this ticket @MartinDupont

MartinDupont commented 2 years ago

I've found the solution. After running apktool d instagram.apk, delete assets/drawables.bin, then run apktool b instagram, and it runs perfectly. (Caveat: I did this with instagram-215-0-0-27-359 with c libraries for a different architecture, but it should work the same). (Caveat 2: With this version, you need to manually delete a line in /res/values/styles.xml, because apktool for some reason generates a doubled item. No big deal).

I don't know what drawables.bin is, but I don't think it's a standard part of an android build. What I think is happening, is that apktool decodes files stored under /r to the /res folder, and upon recompiling, puts them back in the /res folder, and not the original folder, and in addition, it doesn't re-mangle the names. This doesn't normally cause problems for most apps, because the references are still consistent. However, what's probably happening is that drawables.bin is some library which is used to decode the funny-looking drawable declarations, and because we haven't changed it during compile/recompile, it still assumes the resources are sitting under /r with the original names. Which is why it crashes when it tries to load those resources. I don't know why the app still runs when drawables.bin is deleted. This explanation may also be wrong.

In any case, I don't think that apktool has a responsibility to deal with this. Instagram is clearly using a very strange, nonstandard hacky way of getting dynamic resources in their app, and it's clearly a unique case.

This could be solved if apktool was changed so that decompiling and recompiling returned things to exactly the way they were before, but from looking at the apktool code, I think this is too much effort for such a strange edge case.

I vote for closing the issue.

grishka commented 2 years ago

The original problem was that there were multiple references inside resources.arsc to the same file (like @drawable/thing1 and @drawable/thing2 both actually being /res/drawable/thing.png) and apktool couldn't deal with that. I'm not sure whether the problems with current versions of apktool and instagram are caused by this same issue.

MartinDupont commented 2 years ago

I should have been a little more rigorous before. I reran the process with Apktool 2.6.0, Ubuntu 18.04 and Instagram 207.0.0.39.120. I can again get it to recompile and run by just deleting drawables.bin after decompiling.

In any case, the "invalid" drawable declaration that has been generated isn't the problem, as the app runs just fine after being recompiled with those drawables, and they are a faithful copy of what's in the resources.arsc file. The Stacktrace that you get was the same one that I used to get, before I deleted the drawables.bin.

What makes you think that the duplicate references inside resources.arsc are causing an error? And what are the duplicate references? I haven't found any yet

Fmstrat commented 11 months ago

Duplicating here from the chat. I worked around this by decompiling with --no-res, extracting the /r directory from the apk into /unknown in the project, and adding all the files under /r to unknownFiles in apktool.yml. This results in a fully working rebuilt apk.

@grishka Can you give examples of this? What does your unknownFiles section look like? Folder structure? I'm having difficulty understanding what this section of the YML should look like since it's not a standard YML map.

grishka commented 11 months ago 
unknownFiles:
  DebugProbesKt.bin: '8'
  LICENSE_OFL: '8'
  LICENSE_UNICODE: '8'
  billing.properties: '0'
  firebase-annotations.properties: '0'
  firebase-common.properties: '0'
  firebase-components.properties: '0'
  firebase-iid-interop.properties: '8'
  firebase-iid.properties: '8'
  firebase-measurement-connector.properties: '8'
  firebase-messaging.properties: '8'
  play-services-auth-api-phone.properties: '0'
  play-services-auth-base.properties: '0'
  play-services-auth.properties: '0'
  play-services-base.properties: '0'
  play-services-basement.properties: '0'
  play-services-clearcut.properties: '8'
  play-services-flags.properties: '8'
  play-services-location.properties: '8'
  play-services-phenotype.properties: '8'
  play-services-places-placereport.properties: '8'
  play-services-safetynet.properties: '8'
  play-services-stats.properties: '8'
  play-services-tasks.properties: '0'
  play-services-vision-common.properties: '0'
  play-services-vision.properties: '0'
  org/assertj/core/internal/cglib/util/words.txt: '8'
  r/--.xml: "0"
  r/-.xml: "0"
  r/-0.png: "0"
  r/-1.xml: "0"
  r/-2.xml: "0"
  r/-3.png: "0"
  r/-4.9.png: "0"
  r/-5.xml: "0"
  r/-6.xml: "0"
  r/-7.webp: "0"
  r/-8.png: "0"
  r/-9.xml: "0"
  r/-_.xml: "0"
  r/-a.xml: "0"
  r/-b.glsl: "0"
  r/-c.png: "0"
  r/-d.xml: "0"
  r/-e.json: "0"
  r/-f.json: "0"
  r/-g.xml: "0"
  r/-h.png: "0"
  r/-i.png: "0"
  r/-j.xml: "0"
...

The resources themselves appear like they do in the original apk: Снимок экрана 2023-09-29 в 20 44 48 The XML files are binary as well (this particular one must be a state list drawable): Снимок экрана 2023-09-29 в 20 45 48