google / adb-sync

Apache License 2.0
1.09k stars 172 forks source link

Add a flag to preserve time stamps similar to "adb pull -a" #14

Closed divico closed 6 years ago

divico commented 7 years ago

The adb pull command has a -a flag which preserve time stamp. It would be nice if adb-sync had also such a flag.

This is very useful to keep time and date attributes when transferring photos. The gallery app uses these attributes to show the photos in the right order. If time and date attributes are lost during the transfer, the gallery app would show all photos as they were taken on the same day (day of transfer).

divVerent commented 7 years ago

Sounds useful. The reason why it's not there is that I couldn't find a way to push to an Android system while preserving timestamps - supporting this one-way only seems to be a bit odd. Still wouldn't be too opposed to it.

For Photos, wouldn't the Gallery app sort by EXIF timestamp though?

divico commented 7 years ago

Your right that there is no adb push command that preserves time stamps. However, it can be easily achieved by pushing a TAR archive with all the files and there timestamps and decompressing it on the device. Most popular ROMs come with TAR pre-installed.

It would be useful to have the -a flag when adb-sync is used in backup scenario. I doesn't need to be adb-sync default though. Having a -a flag in adb-sync possible options would be enough :)

I think that the stock gallery app does not read EXIF but timestamps. At least its my experience on Lineage 14.1 (Android 7.1). Third apps can update the timestamps from the EXIF, buts its annoying and EXIF is not part of the jpeg standard, so it perfectly valid to have jpeg picture without EXIF, which is the case for WhatsApp images for instance.

divVerent commented 7 years ago

I'll check about the TAR way - I'm just somewhat opposed to having an unidirectional-only flag.

OTOH if I can actually reliably copy the data in both directions, I could massively improve the syncing with that.

On Jun 2, 2017 11:33 AM, "divico" notifications@github.com wrote:

Your right that there is no adb push command that preserves time stamps. However, it can be easily achieved by pushing a TAR archive with all the files and there timestamps and decompressing it on the device. Most popular ROMs come with TAR pre-installed.

It would be useful to have the -a flag when adb-sync is used in backup scenario. I doesn't need to be adb-sync default though. Having a -a flag in adb-sync possible options would be enough :)

I think that the stock gallery app does not read EXIF but timestamps. At least its my experience on Lineage 14.1 (Android 7.1). Third apps can update the timestamps from the EXIF, buts its annoying and EXIF is not part of the jpeg standard, so it perfectly valid to have jpeg picture without EXIF, which is the case for WhatsApp images for instance.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/adb-sync/issues/14#issuecomment-305822678, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPWsL4l6dUNTenysJT6eiG4y_Z0TnLuks5sACs_gaJpZM4Nph3h .

CNG commented 7 years ago

In my personal backup strategy (https://github.com/CNG/android-transfer-media/blob/master/adb/run.sh) before I switched to the Rsync/SSH method, I had resorted to using tar to preserve that data. It worked reasonably well but would need to be made robust against running out of free space and transfer issues. For me, the phone randomly disconnected too much and took too long when I had gigabytes of files in the archive to make that approach useful. That's what led me to this repository!

divVerent commented 7 years ago

Interesting. For me repeatedly running adb push actually makes my Nexus 6 drop the USB connection too much, but works perfectly with my Pixel C.

Using rsync+ssh lets me set a bandwidth limit which seems to work around the Nexus 6 issue. I bet on overheating.

Anyway, once I get to it, I'll see how tar manages to retain the timestamp, and can probably make adb-sync use the same method. Unless it requires root on the device - I don't have a rooted device to test that with.

On Jun 2, 2017 1:23 PM, "Charlie Gorichanaz" notifications@github.com wrote:

In my person backup strategy (https://github.com/CNG/ android-transfer-media/blob/master/adb/run.sh) before I switched to the Rsync/SSH method, I had resorted to using tar to preserve that data. It worked reasonably well but would need to be made robust against running out of free space and transfer issues. For me, the phone randomly disconnected too much and took too long when I had gigabytes of files in the archive to make that approach useful. That's what led me to this repository!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/adb-sync/issues/14#issuecomment-305857114, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPWsMVHS2p6nb0lnbnEtQn1n8kveRn-ks5sAET_gaJpZM4Nph3h .

kwest12 commented 7 years ago

Has anyone come up with a solution? I am trying to backup my entire sdcard since that isn't backed up with a Nandroid, and then I need to move it all over to my new phone. adp pull -a seems to preserve file attributes (dates) but if I'm understanding correctly, adb push doesn't have an equivalent.

Also, adb seems to take issue with certain filenames and just stop the transfer before its complete. Any advice on that matter is appreciated too (happens when trying to pull /Android/data/)

markstock commented 7 years ago

If you can't wait for a solution, you can just edit the script yourself. Here's how I run adb-sync to retain timestamps for pull actions:

diff --git a/adb-sync b/adb-sync
index 53c37c1..9087f6a 100755
--- a/adb-sync
+++ b/adb-sync
@@ -292,7 +292,7 @@ class AdbFileSystem(object):

   def Pull(self, src, dst):
     """Pull a file from the Android device to the local file system."""
-    if subprocess.call(self.adb + [b'pull', src, dst]) != 0:
+    if subprocess.call(self.adb + [b'pull', b'-a', src, dst]) != 0:
       raise OSError('pull failed')
kwest12 commented 6 years ago

So, I'm not nearly advanced enough to understand the method @CNG and @divVerent mentioned (Rsync + SHH?) Is that something that can be run from a Windows computer to sync files the entire sdcard too and from and Android phone? And does it preserve folder structure and file attributes like timestamps and geotags? If so, can someone point me to a (simplistic guide) on how to use it?

@markstock I appreciate whatever you put together there, but I'll be honest, I have very little clue what it does and how to implement it.

I really apologize for coming in with relatively little knowledge on what this stuff is and how to do it. I wish transferring everything off my phone and onto a new one were much easier... When I had to do this in October, I just put a folder at a time into tars, moved them to my computer and then to the new phone, and unpacked them once they were there. I using a relatively full 64GB Nexus 6P, so you can imagine how arduous that process was in order to not exceed the available space (couldn't do it all at once.)

I would really appreciate some help so I don't have to do that again...

markstock commented 6 years ago

@kwest12 My solution will only preserve time stamps on their way off the phone and onto your computer hard drive. If you then want to move those files onto a new phone, the time stamps will not be retained, and that's a limit of adb (I think).

To use my fix, you'll need adb installed (I'm not sure how to do that on Windows). Then clone this git repository and find the adb-sync file. Open it in a text editor and replace the line indicated by the '-' character with the line indicated by the '+'. Save it and run it. I don't do Windows, so I really don't know how to run Python scripts, but be very careful with the command-line arguments. I always run with the "--dry-run" argument first.

divVerent commented 6 years ago

Should be working as far as possible.

kwest12 commented 6 years ago

I see this issue was closed. Does that mean that adb push now preserves timestamps with the -a flag?

divVerent commented 6 years ago

It uses utime - I assume that should be sufficient.

That way we can use the same mode of operation when syncing forward and backward.

On Tue, Nov 27, 2018, 16:47 kwest12 <notifications@github.com wrote:

I see this issue was closed. Does that mean that adb push now preserves timestamps with the -a flag?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/google/adb-sync/issues/14#issuecomment-442229743, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPWsMMkEABluwmQORfO6r7xQF5by2Q7ks5uzbMJgaJpZM4Nph3h .