Closed unclefab closed 10 years ago
Out of memory errors cannot be fixes, they are what they are.
Ok, but I didn't have this problem with previous versions, actually until 1.75...
I guess you have set more restrictions since then.
Mmmmh, I see... So I will investigate, thanks for the info!!!
I have same issue with import. I have 16GB HTC One S with very moderate amount of apps: 199 in total - 55 user apps and 144 system apps, as reported by Titanium Backup. I have around 680MB of free internal storage and around 520MB of free RAM in everyday use. Currently, my XPrivacy xml file size is 303KB (yes, I have a lot of restrictions active). There are 3123 entries for all these apps - an average 15 per app. And each restriction added in XPrivacy app make my file bigger even if I don't change anything. I've tried to reinstall XPrivacy and Xposed, also tried to kill as many apps as possible to free more RAM but nothing helps. My workaround is to Titanium Backup to restore XPrivacy, but others may not have backup. I understand this is not strictly XPRivacy problem, but you might look at this issue. Users who have devices with less memory and/or more apps should be affected as soon as they start do dig deeper and apply more restrictions. I don't know anything about Android development, but have some programming background. My suggestions could be an educated guess at bwest, but here are two ideas:
I will see how I can improve this. Hopefully I don't have to write my own XML parser and can the built-in parser be tweaked.
I will switch to the SAX parser to save memory. http://developer.android.com/reference/javax/xml/parsers/SAXParser.html
Could you please test if this version works okay for you: http://d-h.st/Eyb (if you have a file manager installed, there is a bonus ;-)
See also #327
Same error, but work longer than before.
Marcel Bokhorst notifications@github.com wrote:
Could you please test if this version works okay for you: http://d-h.st/Eyb (if you have a file manager installed, there is a bonus ;-)
Reply to this email directly or view it on GitHub: https://github.com/M66B/XPrivacy/issues/329#issuecomment-21537923
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
There is little more I can do than I already did, sorry.
Maybe change xprivacy to use a sql database. and export the data base as such.
May solve the memory errors and if you locate the database in /data like pdroid it will keep nosy and malicious apps out of it.
That would not be compatible across devices.
Doesnt just about every messaging app use SQL for its database? android includes sql as well as default. and 2.3+ allow the use of the WAL sql database system. It would probably improve the performance alot if this can be done.
Can you explain more why it would not be corss-device compatible? I dont understand why it wouldn't considering my above statements. As for putting it in /data your app will need rood access. but every one who uses exposed framework has to have root anyways. so no issues there.
There are different versions of SQLite database, depending on Android version.
well, you can address it the same way other devs do could you not? You could even include your own version of sql binary. alot of backup apps and tweaking apps do this already.
Too much work that I rather spent on something else (for example restricting Android 4.3)
Fair enough. But If this goes un solved, the inability to import ones settings is going to detour some users and pro users.
I don't have the issue "yet"
But keep in the back of your mind that this issue is fixable with my suggestion of moving to a sql database, and eventually people will expect it to be. I would make this "low prio", not "will not fix" in my opinion.
And again just to mention it would probably improve performance, which i know you have been aiming for.
@GhostlyCrowd An SQLite database doesn't necessarily use less memory. The situation with the new XML parser has greatly been improved. The memory footprint of the new parser is magnitudes lower that the old one.
@pogovault which device are you using and how much free memory do you have?
Also the answer to this stack overflow question hits the nail on the head for the issues your having
http://stackoverflow.com/questions/5213550/raw-resources-versus-sqlite-database/5213569#5213569
easier and faster to retrieve sort and filter data and it has none of the limitations you are getting with SAX or the previous xml parser.
As xprivacy grows so will the things its filtering, sorting and what not. I still suggest you consider this.
As devices become more powerful it will not be uncommon for a user to have 300-600 apps. So what then?
if users are having issues importing settings for 200-300 apps now. thi9nk about the future.
HTC One S, have around 500mb of RAM free.
Marcel Bokhorst notifications@github.com wrote:
@GhostlyCrowd An SQLite database doesn't necessarily use less memory. The situation with the new XML parser has greatly been improved. The memory footprint of the new parser is magnitudes lower that the old one.
@pogovault which device are you using and how much free memory do you have?
Reply to this email directly or view it on GitHub: https://github.com/M66B/XPrivacy/issues/329#issuecomment-21563373
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Here is something i have been able to replicate with xprivacy as well.
I wanted to filter all my apps, including system. I know this breaks android and you boot loop but the purpose to check "All" was so i could then sort and unchecked what i needed to unblock. much faster this way.
So check "All" xprivacy will get an out of memory error and fail to check anything. Check "All" again it will check everything and then xprivacy starts to freeze and lock up. Android asks if you wish to wait for xprivacy and you can, or some times it just blatantly crashes.
I am running cm with about 213 apps including system.
Using the all option just kills xprivacy performance. even with SAX
This doesn't say anything about memory usage when reading/writing XML files (XPrivacy doesn't manipulate the XML files in any other way). XML is the standard to export/import data. Some people edit the exported file manually.
Filtering/sorting has nothing to do with this.
Again: I don't expect problems with the new XML parser, even with a lot of apps, unless the device is already low on memory. The XML parse process as it is now uses hardly any memory.
"... the minimum memory required for a SAX parser is proportional to the maximum depth of the XML file ... and the maximum data involved in a single XML event ..." https://en.wikipedia.org/wiki/Simple_API_for_XML The depth of XML of the XPrivacy XML is the minimum you can get: one and the data involved is almost nothing.
'All' does nothing with the XML file.
then how is the database kept of what you have checked i thought it was all xml? The app keeps a list of your settings i assumed if it wasn't sql it was xml since thats what import and xport use
It uses shared preferences. http://developer.android.com/reference/android/content/SharedPreferences.html
thanks for the clarification, i assumed the fc and freezing was all related to the xml issue. which is why i have not bothered to report it.
I see that in shared_prefs all the apps settings are still xml, they are not affected then by the import xport xml memory issue?
I dunno then, i was hoping switching to sql would increase the performance as well as fix the import export for the users.
SharedPreferences are highly optimized by Google.
Understood.
I tried 7.7.13 beta and still have the problem. Here is an excerpt from logcat: I tried 7.7.13 beta and still have the problem. Here is an excerpt from logcat:
[ 07-25 10:50:29.054 1329: 1346 E/AndroidRuntime ]
FATAL EXCEPTION: pool-1-thread-1
java.lang.OutOfMemoryError: (Heap Size=65543KB, Allocated=64350KB)
at com.android.internal.util.FastXmlSerializer.
[ 07-25 10:50:29.064 965: 976 W/ActivityManager ] Force finishing activity biz.bokhorst.xprivacy/.ActivityMain
Will try later to delete some lines in xml file to find out at what point the error will go away. Will report here.
This is not import/export related, but another problem. Can you try if this version fixes your problem: http://d-h.st/bOW
Yes, http://d-h.st/bOW works!!!! Thanks. By the way, in previous version the biggest file that I was able to import successfully had 259kb (reported by Windows) or 252kb (reported by Android) and had about 2700 lines (entries).
Thanks for reporting back. I am glad this problem is fixed now too.
I just tried 1.7.13 and 1.7.14. Both fail with out of memory errors, fresh from a reboot.
@cobaltdr what where you doing? Can you provide a logcat?
@cobaltdr which device are you using? How much free (not available) memory is there?
build.board: smdk4x12 build.bootloader: I9300XXBLG8 build.brand: samsung build.cpu_abi: armeabi-v7a build.cpu_abi2: armeabi build.device: m0 build.display: JZO54K.I9300XXEMC2 build.fingerprint: samsung/m0xx/m0:4.1.2/JZO54K/I9300XXEMC2:user/release-keys build.hardware: smdk4x12 build.host: SEP-84 build.id: JZO54K build.manufacturer: samsung build.model: GT-I9300 build.product: m0xx build.radio: unknown build.serial: DEFACE <---- ha ha ha build.tags: release-keys build.time: 1362991021000 build.type: user build.user: se.infra version.codename: REL version.incremental: I9300XXEMC2 version.release: 4.1.2 version.sdk_int: 16
07-26 08:00:12.585 E/dalvikvm-heap(25231): Out of memory on a 16400-byte allocation. 07-26 08:00:12.595 E/XPrivacy(25231): java.lang.OutOfMemoryError 07-26 08:00:18.700 E/dalvikvm-heap(25231): Out of memory on a 2359312-byte allocation. 07-26 08:00:18.710 E/XPrivacy(25231): java.lang.OutOfMemoryError
Thanks, but can you give a more complete logging? (more lines, I need to see the stack trace)
Of 832MB that the system listed, about 400MB was unused when I tried to import. Before that, it happened even after a fresh boot.
I noticed that every app it lists as importing takes longer and longer until the crash.
The last time I tried importing, it caused my device to reboot. Unfortunately I wasn't running the logs then. If that happens again I'll try get a logcat.
Gimme a moment.
There doesn't seem to be much there.
07-26 08:00:17.425 I/XPrivacy/XActivityManager(27473): get 1000/getRunningTasks system=false 2 ms
07-26 08:00:17.425 I/XPrivacy/XActivityManager(27473): get 1000/getRunningTasks system=false *
07-26 08:00:17.440 D/dalvikvm(27473): JIT code cache reset in 0 ms (4096 bytes 2/0)
07-26 08:00:17.445 D/dalvikvm(27473): GC_FOR_ALLOC freed 331K, 11% free 16045K/17863K, paused 14ms, total 14ms
07-26 08:00:17.450 I/dalvikvm-heap(27473): Grow heap (frag case) to 19.081MB for 2073616-byte allocation
07-26 08:00:17.470 D/dalvikvm(27473): GC_CONCURRENT freed 12K, 10% free 18058K/19911K, paused 12ms+2ms, total 23ms
07-26 08:00:17.470 D/dalvikvm(27473): WAIT_FOR_CONCURRENT_GC blocked 11ms
07-26 08:00:17.680 D/dalvikvm(2389): GC_FOR_ALLOC freed 129K, 34% free 30423K/46023K, paused 112ms, total 113ms
07-26 08:00:17.705 D/ApplicationPolicy(2389): isStatusBarNotificationAllowed: packageName = com.android.systemui
07-26 08:00:17.725 E/ViewRootImpl(27473): Attempting to destroy the window while drawing!
07-26 08:00:17.725 E/ViewRootImpl(27473): window=android.view.ViewRootImpl@41e13e70, title=
07-26 08:00:17.725 E/CaptureEffectView(27473): onDraw() start
07-26 08:00:17.725 D/STATUSBAR-NotificationService(2389): Noti Alert - mSystemReady:true, AlertEnabled:true
07-26 08:00:17.735 I/SurfaceFlinger(1916): id=104 Removed idx=7 MapSz=5
07-26 08:00:17.740 D/KeyguardViewMediator(2389): setHidden false
07-26 08:00:17.740 I/SurfaceFlinger(1916): id=104 Removed idx=-2 MapSz=5
07-26 08:00:17.760 D/dalvikvm(3089): GC_FOR_ALLOC freed 1706K, 12% free 26192K/29639K, paused 40ms, total 40ms
07-26 08:00:17.770 I/dalvikvm-heap(3089): Grow heap (frag case) to 28.945MB for 2073616-byte allocation
07-26 08:00:17.790 D/dalvikvm(3089): GC_CONCURRENT freed 60K, 12% free 28157K/31687K, paused 2ms+2ms, total 23ms
07-26 08:00:17.790 D/dalvikvm(3089): WAIT_FOR_CONCURRENT_GC blocked 15ms
07-26 08:00:17.790 D/PhoneStatusBar(3089): addNotification score=0
07-26 08:00:17.835 W/ResourceType(3089): Attempt to retrieve bag 0x01030068 which is invalid or in a cycle.
07-26 08:00:17.840 D/dalvikvm(2389): GC_CONCURRENT freed 24K, 30% free 32442K/46023K, paused 3ms+8ms, total 154ms
07-26 08:00:17.840 W/ResourceType(3089): Attempt to retrieve bag 0x01030067 which is invalid or in a cycle.
07-26 08:00:17.860 W/ResourceType(3089): Attempt to retrieve bag 0x010301f2 which is invalid or in a cycle.
07-26 08:00:17.905 D/StatusBar(3089): expanding top notification at 2entry.userCollapsed() = false
07-26 08:00:18.075 I/XPrivacy/XPackageManagerService(2389): get 10031/sdcard storage=false # 64 ms
07-26 08:00:18.110 D/dalvikvm(25231): GC_FOR_ALLOC freed 1563K, 3% free 127323K/131015K, paused 553ms, total 553ms
07-26 08:00:18.110 I/dalvikvm-heap(25231): Forcing collection of SoftReferences for 2359312-byte allocation
07-26 08:00:18.140 I/XPrivacy/XPackageManagerService(2389): get 10031/media storage=false # 65 ms
07-26 08:00:18.205 I/XPrivacy/XPackageManagerService(2389): get 10031/inet internet=false # 64 ms
07-26 08:00:18.225 D/dalvikvm(2389): WAIT_FOR_CONCURRENT_GC blocked 0ms
07-26 08:00:18.370 D/dalvikvm(2389): GC_EXPLICIT freed 1091K, 30% free 32451K/46023K, paused 4ms+8ms, total 143ms
07-26 08:00:18.390 I/XPrivacy(27621): load package=com.android.providers.media uid=10031
07-26 08:00:18.390 I/XPrivacy(27621): Queue usage data=10031/identification/SERIAL size=1
07-26 08:00:18.485 D/dalvikvm(27621): GC_CONCURRENT freed 232K, 10% free 15998K/17735K, paused 12ms+1ms, total 30ms
07-26 08:00:18.650 I/XPrivacy(27621): get 10031/SERIAL identification=false # 260 ms
07-26 08:00:18.660 I/ActivityThread(27621): Pub downloads: com.android.providers.downloads.DownloadProvider
07-26 08:00:18.670 I/XPrivacy(27621): load package=com.android.providers.downloads uid=10031
07-26 08:00:18.670 I/XPrivacy(27621): Queue usage data=10031/identification/SERIAL size=1
07-26 08:00:18.700 D/dalvikvm(25231): GC_BEFORE_OOM freed <1K, 3% free 127323K/131015K, paused 590ms, total 590ms
07-26 08:00:18.700 E/dalvikvm-heap(25231): Out of memory on a 2359312-byte allocation.
07-26 08:00:18.705 I/dalvikvm(25231): "AsyncTask #3" prio=5 tid=15 RUNNABLE
07-26 08:00:18.705 I/dalvikvm(25231): | group="main" sCount=0 dsCount=0 obj=0x4424d380 self=0x149a848
07-26 08:00:18.705 I/dalvikvm(25231): | sysTid=27441 nice=9 sched=0/0 cgrp=apps handle=20907136
07-26 08:00:18.705 I/dalvikvm(25231): | schedstat=( 2284320452 291874473 2280 ) utm=220 stm=8 core=1
07-26 08:00:18.705 I/dalvikvm(25231): at android.graphics.BitmapFactory.nativeDecodeAsset(Native Method)
07-26 08:00:18.705 D/dalvikvm(27621): GC_CONCURRENT freed 504K, 12% free 15916K/17927K, paused 2ms+1ms, total 15ms
07-26 08:00:18.705 I/dalvikvm(25231): at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:623)
07-26 08:00:18.705 I/dalvikvm(25231): at android.graphics.BitmapFactory.decodeResourceStream(BitmapFactory.java:476)
07-26 08:00:18.705 I/dalvikvm(25231): at android.graphics.drawable.Drawable.createFromResourceStream(Drawable.java:781)
07-26 08:00:18.705 I/dalvikvm(25231): at android.content.res.Resources.loadDrawable(Resources.java:1963)
07-26 08:00:18.705 I/dalvikvm(25231): at android.content.res.Resources.getDrawable(Resources.java:672)
07-26 08:00:18.705 I/dalvikvm(25231): at android.content.res.XResources.getDrawable(XResources.java:352)
07-26 08:00:18.705 I/dalvikvm(25231): at android.app.ApplicationPackageManager.getDrawable(ApplicationPackageManager.java:641)
07-26 08:00:18.705 I/dalvikvm(25231): at android.content.pm.PackageItemInfo.loadIcon(PackageItemInfo.java:151)
07-26 08:00:18.710 I/dalvikvm(25231): at biz.bokhorst.xprivacy.ApplicationInfoEx.Initialize(ApplicationInfoEx.java:48)
07-26 08:00:18.710 I/dalvikvm(25231): at biz.bokhorst.xprivacy.ApplicationInfoEx.
How do you go about finding the free memory as opposed to the available memory? I've been using the system's RAM tab after a long press of the home button.
I forgot to mention the XML file is 224KB.
The out of memory exception in the log above is caused while building the application list, not while importing.
You need to install an app to see the free memory. Available memory shown by Android is just the amount of memory your device has (used or not).
That's weird. The out of memory error occurs after the app list is loaded, only when I try to import. When I close my notification tab after the error has occurred, I find XPrivacy updating the app list again. I never have any problem with the app list refreshing, but I've never been able to import a whole XML file without running out of memory. The notification in the notification tab which normally lists the app that is being read from the XML file changes to say out of memory error.
Apparently the import succeeds (I don't see any error message about it in the provide log) and after that there is too little memory to build the application list (to be exact: it fails when rendering the application icons).
Can I zip and pm you the complete log file?
Yes, please do and include the files from /data/data/biz.bokhorst.xprivacy/* if you trust me.
I'm on v 1.77. When I try to import my settings back up file it stops like in the middle of the process and gives me an java.lang.OutOfMemory error.