search

iBotPeaches / Apktool

A tool for reverse engineering Android apk files
https://apktool.org/
Apache License 2.0
20.43k stars 3.6k forks source link

Error parsing XML: not well-formed (invalid token) #885

Closed iBotPeaches closed 7 years ago

iBotPeaches commented 9 years ago

Original issue 776 created by kefir500 on 2015-03-04T10:57:38.000Z:

What steps will reproduce the problem?

What is the expected output? What do you see instead?

warning: string 'pref_map_show_center_cross' has no default translation.
warning: string 'pref_map_show_center_cross_desc' has no default translation.

C:\Apktool\res\layout\abc_action_menu_layout.xml:2: error: Error parsing XML: not well-formed (invalid token)

C:\Apktool\res\layout\abc_action_mode_bar.xml:2: error: Error parsing XML: not well-formed (invalid token)

C:\Apktool\res\layout\abc_activity_chooser_view.xml:2: error: Error parsing XML: not well-formed (invalid token)

--- And 5 more similar errors ---

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_text' in package 'android'

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_textSize' in package 'android'

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_barColor' in package 'android'

-- And >10 similar errors ---

C:\Apktool\res\layout\view_color_picker_basic_tab.xml:5: error: Error parsing XML: duplicate attribute

What version of the product are you using? On what operating system?

Please provide any additional information below. Seems that XML tags are corrupt. Tried to fix these XML tags manually - no luck. Also tried different aapt and Apktool releases - still the same. Removing "framework/1.apk" does not help.

iBotPeaches commented 9 years ago

Comment #1 originally posted by connor.tumbleson on 2015-03-11T12:25:18.000Z:

Yeah no idea what this is. There are some strange characters in the layout files.

<o.ᴗ android:gravity

Dumping out the resources via aapt

ibotpeaches@raganok:~/Downloads/Apktool/Bug776$ aapt d xmltree locus.apk res/layout/abc_action_menu_layout.xml N: android=http://schemas.android.com/apk/res/android N: app=http://schemas.android.com/apk/res-auto E: o.ᴗ (line=17)

Yields the same weird character. Needs more investigation.

iBotPeaches commented 9 years ago

Comment #2 originally posted by Liquid.St.Ray on 2015-03-13T23:20:00.000Z:

The unknow tag name is actually the full qualified name of a custom UI class, which has been obfuscated. I suffered the same issue and I found the symbol in smali files. I think this could be a common case for lots of commerical apps.

iBotPeaches commented 9 years ago

Comment #3 originally posted by connor.tumbleson on 2015-03-13T23:21:51.000Z:

Is this from a certain ROM/OEM then? How does an obfuscated resource work without being de-obfuscated prior to use?

Its not like Android knows this mapping of obfuscation -> real resource names.

iBotPeaches commented 9 years ago

Comment #4 originally posted by Liquid.St.Ray on 2015-03-13T23:52:04.000Z:

Should be some unexpected characters was introduced in the obfuscation and AAPT is complaining about it.

I tried to rename a custom UI component class using unexpected characters and suffered the same issue when building the APK file.

That APK could be packed with custom built AAPT I think.

iBotPeaches commented 9 years ago

Comment #5 originally posted by Liquid.St.Ray on 2015-03-14T00:18:07.000Z:

DexGuard could generate non-ASCII characters for class/method/field names, which is not supported by stock AAPT.

iBotPeaches commented 9 years ago

Comment #6 originally posted by connor.tumbleson on 2015-03-14T00:20:05.000Z:

aapt is all about resources. It doesn't manage the source (java) at all, so those points aren't relatable.

iBotPeaches commented 9 years ago

Comment #7 originally posted by Liquid.St.Ray on 2015-03-14T00:25:04.000Z:

You can claim a custom UI class (TextView for example) and reference it in the layout xml file. In this case it would involve compiled resoruce file.

Once that class was obfuscate by something like DexGuard, the class reference in layout xml file would be modified as well.

iBotPeaches commented 9 years ago

Comment #8 originally posted by connor.tumbleson on 2015-03-14T00:50:24.000Z:

Ah true true. Custom UI classes then are just localized mapping in an application. So it doesn't matter if the naming is Foo or some crazy UTF8 character.

The question remains AAPT is obviously not liking it. So as you said a custom AAPT must be used. With that being said, do you have any information on this? I looked around there website (Dexguard/Proguard) and couldn't find anything.

iBotPeaches commented 9 years ago

Comment #9 originally posted by alkorware on 2015-03-14T18:33:01.000Z:

it's 100% dexguard's obfuscation.

ddccffvv commented 8 years ago

Would it be possible to work around this by just renaming the classes to unique [a-Z*] names and updating the references (during/after decompilation)?

jack16099 commented 7 years ago

Is this issue fixed? I think I've seen some modded apps which gives the same error on latest version of apktool and there's those illegal characters in the manifest. If I'm not wrong that it's modded, how did they do it if apktool can't?

warren-bank commented 4 years ago

though this issue is closed, I recently encountered this behavior with apktool 2.4.1

to summarize, the problem traced back to the fact that the decoded apk contained smali filenames with unicode characters.

as a workaround, I wrote a small nodejs command-line utility to rename all smali unicode filenames, and remap all references elsewhere in the project.

wrote it in a day.. might not solve everyone's woes, but I hope that some may find it useful; worked for me.

pgp commented 3 years ago

Same issue here, I tested the same apk (titanium backup 7.6.0.1) linked in https://github.com/iBotPeaches/Apktool/issues/1389 , using apktool 2.5.0 ... Gonna try @warren-bank script for now

warren-bank commented 3 years ago

I'm in a virtualized environment right now, so I can't check my notes (on modding TB) until later.. but I do remember TB having some unique issues. I believe that more recent versions decompile with missing resource directories, in addition to unicode issues. I had to iteratively test older versions until I found the last one to decompile without any missing resource directories. It was that version that I was able to successfully round-trip: decompile, (optionally) modify smali source files, run my script, recompile, and run in Android without any installation or runtime errors.

pgp commented 3 years ago

Actually, on another apk that had this issue, I managed to make recompile work (and also to correctly run on the emulator the recompiled apk after resigning it) by replacing: <id with: <item type="id" in the res/values/ids.xml file. On the above mentioned titanium backup apk, the inconsistent files I edited so far using this strategy are:

res/values/ids.xml
res/values/layouts.xml
res/values/drawables.xml

but now apktool complains about res/values/colors.xml, it seems because in this case the xml tag has no content (apart from the name property). Still, it is unclear to me why apktool generates these xml files with inconsistent schema.

Disclaimer: what I'm trying to do is recompiling some apks I don't trust, after having removed INTERNET permission from manifest (this avoids the burden of installing 3rd party firewalls for having the same functionality, since android doesn't have a decent way of providing firewall feature at the OS level)

warren-bank commented 3 years ago

As an aside.. we should really set up some kind of repo:

I don't know about any of you, but:

pgp commented 3 years ago

Actually, I started experimenting with apktool just for curiosity, so that when I found those apk that gave errors during recompilation, I also tried to reproduce this by using an obfuscation dictionary in proguard rules in one application of mine: I thought these errors were due to usage of non-ascii characters in proguard mappings, yet the apk generated for my application decompiled and recompiled fine even with the proguard rules modification. Now I'm even more curious to know how to reproduce this scenario in my source code, so any hint of possible proguard settings responsible of this issue is welcome.

pgp commented 3 years ago

BTW: you were talking about modifying smali files, it's clearly not my case, so this issue should be related to something else at least.