IzzySoft / Adebar

Android DEvice Backup And Report, using Bash and ADB. Moved to https://codeberg.org/izzy/Adebar -- this is only a mirror now.
GNU General Public License v2.0
246 stars 41 forks source link

Root: some files are not found #3

Closed tukusejssirs closed 9 years ago

tukusejssirs commented 9 years ago

I pulled new devel branch repository and enable PARTBACKUP setting in config/bu file, but after I ran ./adebar bu, some files cannot be found. Here’s the latest output (I enabled some more verbosity in the config/bu file):

./adebar-cli bu

Adebar running:
Gathering lists of installed apps
- userApps
- systemApps
- checking for uninstalled apps remembered by the system
Removed app: 
Removed app: 
Removed app: 
Removed app: 
Removed app: 
Removed app: 
Removed app: 
Removed app: 
Removed app: com.gameloft.android.GloftRF14
Removed app: com.accuweather.android
Removed app: com.facebook.katana
Removed app: com.gameloft.android.GloftGF2H
Removed app: com.gamestore
Removed app: com.mediatek.FMRadio
Removed app: com.UCMobile.intl
Removed app: com.evernote
Removed app: com.route66.maps5
Removed app: com.gameloft.android.gdc
Removed app: com.guvera.android
Removed app: com.twitter.android
Removed app: com.skype.raider
Removed app: com.gameloft.android.HEP.GloftA7HP
Removed app: com.txtr.android.lenovo
Obtaining list of disabled apps
Creating backup/restore scripts for UserApps
Creating backup/restore scripts for SytemApps
Checking default install-location
Collecting partition details
Generating script to create partition images
Collecting device information
- Evaluating device properties
   + checking for product details
   + checking for sensor details
   + checking for OS details
- Evaluating 'pm list features'
- Evaluating device status
  + Battery status
  + Radio status
  + networking details
  + storage details
  + Provider info
- Obtaining account and backup details
Pulling configuration files
- build.prop: 99 KB/s (4571 bytes in 0.044s)
- wpa_supplicant.conf: remote object '/data/misc/wifi/wpa_supplicant.conf' does not exist
- softap.conf: remote object '/data/misc/wifi/softap.conf' does not exist
- gps.conf: remote object '/system/etc/gps.conf' does not exist
- hosts: 3930 KB/s (599678 bytes in 0.148s)
- packages.xml: failed to copy '/data/system/packages.xml' to 'bu/conf/packages.xml': Permission denied
- usage-history.xml: remote object '/data/system/usagestats/usage-history.xml' does not exist
Generating app detail info
- collecting data via dumpsys
- parsing package dump
Oops in pkg at line 229: Unexpected: '    mtkFlags=[ ]'
- generating userApps.md
- removing temporary dumpfile
PostProcessing and Cleanup
Adebar done.

Following files do exist in my system

File /system/etc/gps.conf does not exist. This might be because I never had gps turned on. :)

In the config/bu file, there are some settings (read: variables :) …) which I don’t really understand what exactly they do. For example, there’s the following setting:

# If your device is rooted, but the ADB daemon does not run in root mode, set this to 1
ROOT_COMPAT=0

I have a rooted phone and I can access root shell in adb (in two steps, adb shell and su). I just don’t know if I should enable it or not.

tukusejssirs commented 9 years ago

I am blind or something … :/ … The partBackup is there. My bad.

But all the other issues are still relevant. I’ll just rename this issue to something more appropriate. :)

partBackup (As for github does not allow me to upload here the file directly, i just appended the .jpg extension, but still it’s a text file. You can download it by right-clicking on it.)

IzzySoft commented 9 years ago

Try setting ROOT_COMPAT=1 (I should make that the default, maybe).

Background: To obtain those files, Adebar uses adb pull – which does not run with root privileges unless the ADB daemon does, and thus cannot access several files. Turning "root compatibility mode" on, Adebar works around that obtaining those files using a different method.

Thanks for attaching the partPackup file – unfortunately it seems Git doesn't want to be tricked. Instead of showing/downloading the file, it simply displays an error message in the browser. Maybe you just put it on pastebin.com and link here?

tukusejssirs commented 9 years ago

Well, of course it displays that error while left-clicking on the link, because github assumes it’s a real .jpg file. However, I can’t see why are you not able to download the file from here – you need to right-click the link, clik on Save Link As, rename it to partBackup and you’re good to go. :)

Anyway, it is also on pastebin.

IzzySoft commented 9 years ago

Wonderful – thanks! So we can count /pric/dumchar_info as "working as expected" (and I can remove the "testing needed" remarks on that – done at 27a5d88) – apart from line 28, which should not be there. Could you please pastebin output of adb shell cat /proc/dumchar_info?

How did the "root compat" change work out for you? Files obtained now, so we can close this issue?

tukusejssirs commented 9 years ago

Well, I think it does nothing actually, at least not for me. :/

Here is the list of all my files after the latest backup:

$ find .
.
./userrestore
./userbackup
./docs
./docs/deviceInfo.md
./docs/sysApps.md
./docs/userApps.md
./defaultInstallLoc
./sysrestore
./conf
./conf/build.prop
./conf/hosts
./deadReceivers.sh
./sysbackup
./disable

There is now wpa_supplicant.conf file, for example.

IzzySoft commented 9 years ago

Adebar output would help: any errors displayed – and if so, which? If you're using your own config file meanwhile, increase the value of PROGRESS (e.g. to 9 – there's no level that high, but it makes sure all info/debug messages are printed).

tukusejssirs commented 9 years ago

The adebar output is as follows:

./adebar-cli progress/
Adebar running:
Gathering lists of installed apps
Obtaining list of disabled apps
Creating backup/restore scripts for UserApps
Creating backup/restore scripts for SytemApps
Checking default install-location
Collecting partition details
Collecting device information
Pulling configuration files
Generating app detail info
PostProcessing and Cleanup
Adebar done.

No errors. The output is the same if PROGRESS variable is either 3 or 9, it does really matter (as far as I undertand it correctly, 3 is currently the highest verbosity set by you as a programmer).

IzzySoft commented 9 years ago

The output in your last post reflects PROGRESS=1 – and the command you've executed (./adebar-cli progress/) made sure no config was used. As described, you must pass only the name of the config file as parameter (e.g. if your config file is config/bu, the call must be ./adebar-cli bu – no trailing slash, and the config file must reside in the config/ directory).

In fact, the highest verbosity level used is currently 5 – but everything above 3 is rather debug. For your missing files, level 2 should be enough – and we then need the output immediately following Pulling configuration files.

tukusejssirs commented 9 years ago

Okay, now I probably got it.

I need to set STORAGE_BASE variable to the directory in which I want the backup and then after issuing the command ./adebar-cli bu the backup will be there.

Actually, now it works. Thank you for your patience with me.

Just one more question. How successfull is it in restoring a backup? Let’s say I make a factary restore, flash custom recovery (TWRP), root my device and restore the backup from ./adebar, will be everything working just like it is now, or would there be some need of tweak?

tukusejssirs commented 9 years ago

Well, actually there’s one error in the output:

$ ./adebar-cli bu

Adebar running:
Gathering lists of installed apps
- userApps
- systemApps
- checking for uninstalled apps remembered by the system
  + com.gameloft.android.GloftRF14
  + com.accuweather.android
  + com.facebook.katana
  + com.gameloft.android.GloftGF2H
  + com.gamestore
  + com.mediatek.FMRadio
  + com.UCMobile.intl
  + com.evernote
  + com.route66.maps5
  + com.gameloft.android.gdc
  + com.guvera.android
  + com.twitter.android
  + com.skype.raider
  + com.gameloft.android.HEP.GloftA7HP
  + com.txtr.android.lenovo
Obtaining list of disabled apps
Creating backup/restore scripts for UserApps
Creating backup/restore scripts for SytemApps
Checking default install-location
Collecting partition details
Generating script to create partition images
Collecting device information
- Evaluating device properties
   + checking for product details
   + checking for sensor details
   + checking for OS details
- Evaluating 'pm list features'
- Evaluating device status
  + Battery status
  + Radio status
  + networking details
  + storage details
  + Provider info
- Obtaining account and backup details
Pulling configuration files
- build.prop: 110 KB/s (4571 bytes in 0.040s)
- wpa_supplicant.conf: OK
- softap.conf: OK
- gps.conf: remote object '/system/etc/gps.conf' does not exist
- hosts: 3839 KB/s (599678 bytes in 0.152s)
- packages.xml: OK
- usage-history.xml: OK
Generating app detail info
- collecting data via dumpsys
- parsing package dump
  - Found package list marker
  - Package: com.germainz.pinnotif
Oops in pkg at line 229: Unexpected: '    mtkFlags=[ ]'
  - Package: com.android.defcontainer
  - Package: com.pyler.nosafevolumewarning
  - Package: com.android.phone
  - Package: com.sand.airdroid
  - Package: com.alensw.PicFolder
  - Package: com.ekledesigns.cmd_calc
  - Package: dn.mansi.iis
  - Package: com.android.bluetooth
  - Package: com.android.providers.calendar
  - Package: com.lenovo.lsf.device
  - Package: com.google.android.apps.docs.editors.sheets
  - Package: com.aor.droidedit.pro
  - Package: com.mediatek.voicecommand
  - Package: com.gameloft.android.GloftRF14
  - Package: com.android.providers.downloads.ui
  - Package: com.contapps.android
  - Package: com.android.documentsui
  - Package: com.android.vpndialogs
  - Package: com.android.mms
  - Package: com.accuweather.android
  - Package: com.magicandroidapps.bettertermpro
  - Package: com.smartmadsoft.xposed.s5.nowakeoncharge
  - Package: com.lenovo.themecenter
  - Package: com.android.providers.media
  - Package: com.byril.seabattle2
  - Package: uk.co.rogueideas.barcodearchitect
  - Package: pl.suzume.xposed.disabler
  - Package: com.mediatek.apkinstaller
  - Package: com.mediatek.bluetooth
  - Package: com.nadeemsultan.hidehandle
  - Package: com.google.android.setupwizard
  - Package: com.lenovo.ota
  - Package: com.p1ngu1n.volumesteps
  - Package: com.fw.upgrade.sysoper
  - Package: com.android.settings
  - Package: jackpal.androidterm
  - Package: com.fw.upgrade
  - Package: com.mediatek.thermalmanager
  - Package: com.mediatek.calendarimporter
  - Package: com.facebook.katana
  - Package: com.android.providers.drm
  - Package: com.boblc.disablechargingled
  - Package: com.gameloft.android.GloftGF2H
  - Package: com.germainz.stopswitchdelay
  - Package: com.android.exchange
  - Package: sk.mam.skento
  - Package: com.google.android.backuptransport
  - Package: com.android.providers.telephony
  - Package: com.mediatek.CellConnService
  - Package: com.svox.pico
  - Package: com.pyler.nodevicecheck
  - Package: com.gamestore
  - Package: com.android.dialer
  - Package: com.android.ServiceMenu
  - Package: com.anddoes.launcher
  - Package: com.android.wallpapercropper
  - Package: com.android.providers.settings
  - Package: com.offline.ocr.english.image.to.text
  - Package: com.aor.droidgrep
  - Package: com.n0n3m4.droidsdl
  - Package: com.android.providers.downloads
  - Package: com.mediatek.FMRadio
  - Package: com.android.browser.provider
  - Package: com.beatslayer.xposedmod.netstrength
  - Package: com.mediatek.vlw
  - Package: com.google.android.onetimeinitializer
  - Package: sk.tb.ib.tatraandroid
  - Package: com.bocharov.xposed.fsbi
  - Package: org.cabagge.knizka
  - Package: com.tplink.tether
  - Package: de.defim.apk.hidebatterylowalert
  - Package: com.android.inputdevices
  - Package: stericson.busybox
  - Package: us.porrassoft.volumebooster
  - Package: com.mohammadag.screenshotdelayremover
  - Package: de.j4velin.notificationToggle
  - Package: com.android.stk
  - Package: com.UCMobile.intl
  - Package: com.google.android.configupdater
  - Package: com.macropinch.swan
  - Package: com.evernote
  - Package: com.mediatek
  - Package: com.lenovo.ue.service
  - Package: android
  - Package: com.android.providers.contacts
  - Package: com.germainz.crappalinks
  - Package: com.android.externalstorage
  - Package: com.android.ServiceMenu3
  - Package: com.android.providers.applications
  - Package: com.google.android.apps.translate
  - Package: me.piebridge.forcestopgb
  - Package: com.android.ServiceMenu2
  - Package: com.n0n3m4.droidc
  - Package: com.ghisler.android.TotalCommander
  - Package: com.android.vending
  - Package: com.coletz.clocktoflash
  - Package: com.android.simmelock
  - Package: com.android.keychain
  - Package: org.mozilla.firefox_beta
  - Package: eu.chainfire.supersu
  - Package: com.route66.maps5
  - Package: com.mediatek.engineermode
  - Package: org.pocketworkstation.pckeyboard
  - Package: com.google.android.gsf
  - Package: com.android.keyguard
  - Package: com.sysinfodroid
  - Package: com.googlecode.eyesfree.setorientation
  - Package: com.android.shell
  - Package: com.mediatek.appguide.plugin
  - Package: cz.pajus.bibliaslovenska
  - Package: com.google.android.syncadapters.contacts
  - Package: com.mediatek.security
  - Package: de.defim.apk.notifyclean
  - Package: de.robv.android.xposed.installer
  - Package: com.android.contacts
  - Package: com.android.providers.partnerbookmarks
  - Package: com.mediatek.lbs.em
  - Package: com.android.calculator2
  - Package: com.android.htmlviewer
  - Package: com.google.android.gsf.login
  - Package: org.ebookdroid
  - Package: com.mediatek.mtklogger
  - Package: com.thetonyp.batteryshutdownmanager
  - Package: com.gh4a
  - Package: com.android.sharedstoragebackup
  - Package: com.opera.mini.native
  - Package: hk.kennethso168.xposed.apmplus
  - Package: com.mediatek.filemanager
  - Package: com.ceco.sbdp
  - Package: com.farproc.wifi.analyzer
  - Package: com.google.android.marvin.talkback
  - Package: com.android.certinstaller
  - Package: com.mediatek.connectivity
  - Package: hdn.android.countdown
  - Package: com.keramidas.TitaniumBackup
  - Package: com.pyler.xinstaller
  - Package: com.google.android.gms
  - Package: net.sourceforge.opencamera
  - Package: ba.makrosoft.mega
  - Package: org.simalliance.openmobileapi.service
  - Package: de.blinkt.vpndialogxposed
  - Package: com.android.gallery3d
  - Package: com.gameloft.android.gdc
  - Package: com.mediatek.DataUsageLockScreenClient
  - Package: com.android.packageinstaller
  - Package: ru.org.amip.ClockSync
  - Package: com.android.email
  - Package: org.videolan.vlc
  - Package: com.android.location.fused
  - Package: com.android.backupconfirm
  - Package: com.mediatek.op02.plugin
  - Package: ru.org.amip.timezoneservice
  - Package: sk.breviar.android
  - Package: com.google.android.apps.docs.editors.docs
  - Package: com.android.musicfx
  - Package: com.android.soundrecorder
  - Package: com.shubhangrathore.xposed.disablefullscreenkeyboard
  - Package: com.hamzah.lockscreentweakbox
  - Package: ztc1997.vibrationtuner
  - Package: com.android.inputmethod.latin
  - Package: com.google.android.partnersetup
  - Package: fr.xplod.focal
  - Package: com.mediatek.omacp
  - Package: com.android.proxyhandler
  - Package: org.zwanoo.android.speedtest
  - Package: in.proficientapps.gmte.lite
  - Package: com.guvera.android
  - Package: com.google.android.feedback
  - Package: com.android.providers.userdictionary
  - Package: com.dic_o.dico_svk_ger
  - Package: com.mediatek.apst.target
  - Package: com.hamzah.dimenseditor
  - Package: com.androguide.ryuinferno.universal.init.d.support
  - Package: com.twitter.android
  - Package: pk.qwerty12.receivedsmssenttimesindetails
  - Package: com.android.pacprocessor
  - Package: biz.underpants_gnomes.android.xposed.mods.hidexposedime
  - Package: com.skype.raider
  - Package: com.cleanmaster.mguard
  - Package: com.viber.voip
  - Package: org.joa.zipperplus7
  - Package: sk.tb.emv.mobile
  - Package: com.google.android.syncadapters.calendar
  - Package: com.elelinux.ubuntu
  - Package: com.gameloft.android.HEP.GloftA7HP
  - Package: com.android.dreams.basic
  - Package: com.n0n3m4.gcc4droid
  - Package: nz.mega.android
  - Package: com.android.systemui
  - Package: com.force.timezonefixer
  - Package: uk.co.villainrom.pulser.disablebatteryfullalert
  - Package: com.google.android.gm
  - Package: de.defim.apk.enhancedtoast
  - Package: com.txtr.android.lenovo
  - Package: lv.id.dm.hostsenabler
  - Package: com.byd.runin
  - Package: com.spartacusrex.spartacuside
  - Package: com.android.deskclock
  - Package: com.ghisler.tcplugins.LAN
  - Package: com.apex.minimal
  - Package: com.beatslayer.xposedmod.notifications
  - Package: com.google.android.calendar
  - Package: de.defim.apk.bootmanager
  - Package: com.mediatek.ygps
  - Package: com.thomashofmann.xposed.mediascanneroptimizer
- collecting app usage stats
- generating userApps.md
- removing temporary dumpfile
PostProcessing and Cleanup
Adebar done.
IzzySoft commented 9 years ago

How successfull is it in restoring a backup?

Adebar itself doesn't do much backup/restore (though it pulls some stuff, that's more for reference/documentation) – it generates the scripts for you so you can backup/restore, saving you from doing that manually. Remember, when you do a complete backup via ADB, you also can do a complete restore only (no picking of separate apps). Adebar instead generates scripts to backup everything separately, so you can pick what to restore. So how well backup/restore itself works is up to ADB.

I've used those scripts myself a couple of times. Always adjusted little things manually (mostly to skip stuff I didn't want to restore). Had no issues so far.

Well, actually there’s one error in the output:

Talking about the "Oops"? "Oopses" are just pointers that there is stuff I didn't encounter yet, where more information might be obtained. You might have noticed that one again is MTK specific (remember, I have no MTK device). If you're interested in that, I suggest we deal with it in a separate issue – as everything now seems to work fine (and your output seems to indicate that), I'd prefer to close this issue – so I can go releasing v1.3.0, and then we can investigate new things (like your Dual-SIM feature – or this "oops".

If that's fine with you, and the original issue solved: Please close, so we can go on :)

tukusejssirs commented 9 years ago

Restoration

So you mean that after making a restore, it would look like it aws before factory reset (of course, with keeping in mind the known issues)? Actually, I think there are no settings backed up, are they? (I mean system-wide settings.)

I know that adb's backup is either all or nothing. Still, adebar another advantage – the backup’s smaller in size, though the downside that restoration require connection to the Internet and therefore is as fast as the connection is.connection

Oops

Okay, no problem with that. Yes, indeed it is an MTK specific one. Okay, I created new issue here.

tukusejssirs commented 9 years ago

I’d like to closed #3 issue, but I am not able to do so – probably only developer or contributor can do so.

IzzySoft commented 9 years ago

Actually, I think there are no settings backed up, are they? (I mean system-wide settings.)

Not many. Mainly those config files. Everything else goes via ADB – which is not a complete backup (adb backup e.g. has no access to your SMS/MMS and a few other things, plus apps can "opt-out" of it by setting ALLOW_BACKUP to "No" – but that's nothing Adebar can do anything about.

the backup’s smaller in size

I never calculated that. As said, Adebar only creates the scripts you must run in order to backup stuff – I've never compared resulting file sizes, but as those scripts doing nothing but utilizing adb backup, I'd wonder if that results in smaller size.

though the downside that restoration require connection to the Internet

Huh? What for? It can use "the Internet" to retrieve app names – but I don't see where it requires an Internet connection for restoring stuff. The restore scripts again are nothing but calls to adb backup.

I’d like to closed #3 issue, but I am not able to do so – probably only developer or contributor can do so.

And the one who opened it – I did that many times on other projects I'm not part of. But no problem, as you indicated you agree to that, I can do it now. Thanks a lot for your collaboration! I'll go and make the next release then (soon, currently I've got some other tasks to absolve).