search

bepaald / signalbackup-tools

Tool to work with Signal Backup files.
GNU General Public License v3.0
790 stars 38 forks source link

Importing desktop messages twice adds all messages twice #255

Open h0jeZvgoxFepBQ2C opened 1 day ago

h0jeZvgoxFepBQ2C commented 1 day ago

Hi,

today I successfully restored an old backup (1 year old) together with my desktop backup (up to date), but now I have all messages twice included (since I did the import twice), due to following reasons.

My procedure was:

1) download old backup 2) download desktop backup 3) import desktop data to old backup (with warnings, that contacts couldnt be found) 4) import restored backup to phone 5) start a new backup on my phone 6) enrich the new backup again with the desktop backup again (this time no contact not found errors) 7) works fine, all groups / messages back again - but all messages are now doubled, also all photos

Is there any way to prevent the duplication of attachments + messages? Don't they have an ID, which could get skipped in case its already existing?

Thank you so much for your hard work, this program really saved me a lot today! ❤️

bepaald commented 1 day ago

Hi!

Is there any way to prevent the duplication of attachments + messages?

Normally, when importing from desktop, the way to limit which messages get imported is by --limittodates/--autolimitdates. In your case, simply importing the same desktop data twice, this probably won't work because the date-ranges of the messages imported at each pass overlap.

Don't they have an ID, which could get skipped in case its already existing?

No they don't. In your specific case I could probably skip importing when messages have the same body, to- and from-, and timestamp. Normally this does not work since even the timestamps between Android and Desktop differ, but in your case since the duplicates both come from the same Desktop data, their timestamps will be identical. Alternatively, I could implement a --limittothreads option, that would allow you to only import specific threads when running the second --importfromdesktop.

However, neither of those options is currently available, because really you should not ever need to run --importfromdesktop twice in a row like this. In your procedure:

  1. download old backup
  2. download desktop backup
  3. import desktop data to old backup (with warnings, that contacts couldnt be found)
  4. import restored backup to phone
  5. start a new backup on my phone
  6. enrich the new backup again with the desktop backup again (this time no contact not found errors)
  7. works fine, all groups / messages back again - but all messages are now doubled, also all photos

I really don't understand why step 3 is there. Importing from desktop certainly did not cause your missing contacts to be synced. I think you should just skip that step: restore the old backup (without imported desktop data), maybe wait a minute or so (for contacts to sync), and export a new backup, and then run --importfromdesktop on that (once, optionally with one of the date limiters mentioned above, in case of overlap between the old backup and the Desktop data). Or had you tried that before, and does it not work?

Thank you so much for your hard work, this program really saved me a lot today! ❤️

Thanks, I'm happy things are looking good so far. Let me know if the above suggestions work for you, or if I need to add some functionality.