SMS import does not work

sudwhiwdh commented 1 year ago

Good day,

the following steps to reproduce:

Install Simple SMS Messenger in version 5.18.1
Write some messages with existing imported contacts from Simple Contacts application. As well as with some new numbers/contacts not yet saved.
Select message export in the menu. The following appears:
```
Exporting...
```

Exporting successful


4. Uninstall/reinstall the app or select and delete all messages.

5. Select message import in the menu.
The following appears:

Importing...

Error: java.lang.IllegalStateException: Expected a name, but it was END_OBJECT in line 1 column 251 path $[0].sms

Importing some entries failed



Only one single SMS could be recovered from all exported conversations and messages.

The file size is 22kB.
However, I have had similar experiences with exported Simple SMS Messenger backups smaller than 5MB.

All attempts always end up with the same error message and the loss of my messages and data.

sudwhiwdh commented 1 year ago

I have chosen "SMS import does not work" for the issue title. However, I think something is already not working as it should when exporting.

sudwhiwdh commented 1 year ago

Is there currently any validation of the .json file at the end of the export process before Exporting successful is displayed?

tibbi commented 1 year ago

not really, there isnt.

sudwhiwdh commented 1 year ago

Would it be possible for you to include an automatic JSON validation test here, which is even able to correct possible errors before the export, so that it is likely that the file can be imported again without errors?

tibbi commented 1 year ago

fixing the issue itself would be more helpful..

sudwhiwdh commented 1 year ago

Do you already have an idea how to do it?

sudwhiwdh commented 1 year ago

The validation check would also be more of an additional idea to detect and repair so far undiscovered issues before they lead to data loss for users.

sudwhiwdh commented 1 year ago

I had a moment to test a little. Tried the export function again with the newly arrived SMS in the database.

Export with Simple-SMS-Messenger with SMS & MMS check mark.
Export with Simple-SMS-Messenger with SMS checkmark.
Export with SMS Import / Export application from F-Droid.

Then I tried to restore all three files again using the import function of Simple-SMS-Messenger.

The result: When restoring the .json files that I had previously exported with Simple-SMS-Messenger, even with a small number of messages (~100 in that case), the error message mentioned in 5. appears.

The interesting thing is that only one conversation is restored at a time when importing files created with Simple SMS Messenger. And this is the conversation in which I last wrote an SMS and which is therefore at the beginning of the .json file.

The .json file I created with SMS Import / Export gives a slightly different error:

Importing...

Error: java.lang.IllegalStateException:
Expected a name, but it was STRING in
line 3 column 13 path $[0]._id

No entries found.

I also tried to see if restarting the device or turning it off and on or restarting Simple SMS Messenger would change the amount of imported data. It does not.

sudwhiwdh commented 1 year ago

@tibbi Does Simple SMS Messenger create log files that would show where the error might be when exporting or importing the SMS messages?

tibbi commented 1 year ago

not really, the exporting maybe stops for some reason and the file isnt closed properly.

sudwhiwdh commented 1 year ago

not really

You mean Simple SMS Messenger doesn't create a log file or you don't have an idea how to solve the problem yet?

tibbi commented 1 year ago

I dont see how could any log file help in this issue

sudwhiwdh commented 1 year ago

I performed a fourth test with QKSMS,

because this application also exports the messages in .json.

https://f-droid.org/en/packages/com.moez.QKSMS/

The result was that the messages exported from QKSMS could be imported back into QKSMS without any problems.

When trying to import the same .json file into Simple-SMS-Messenger the following error message appeared:

Importing...

Error: java.lang.IllegalStateException:
Expected BEGIN_ARRAY, but it was BEGIN_OBJECT in
line 1 column 2 path $

No items found.

sudwhiwdh commented 1 year ago

I was able to import the data again after updating to https://github.com/SimpleMobileTools/Simple-SMS-Messenger/releases/tag/5.18.2. @kcotugno Thanks for the fix!

And still see room for improvement:

The import for a ~2MB file took ~40 minutes, which already seems very long to me. How come export takes ~3/4 minutes and import takes ten times longer?
I would like to see the progress displayed: How many messages or conversations exist in total, how many are already imported/still open. Perhaps also an indication of how long the import will take to complete. Of course, the same would be nice for the export dialogue as well.
The import dialogue cannot be closed. Confirming with OK does nothing.
Messages should be sorted by their timestamp/jsonID?, not by their import time.
I don't remember for sure if there is a visual notification when the import is finished. I was not watching the display for the whole 40 minutes. Maybe it makes sense to throw out a system notification instead of an infobar within the app?

tibbi commented 1 year ago

that definitely shouldnt have taken 40 minutes... Didnt you have some other file operations in the background?

sudwhiwdh commented 1 year ago

that definitely shouldnt have taken 40 minutes... Didnt you have some other file operations in the background?

I am quite sure that there were no other file operations in the background.

kcotugno commented 1 year ago

I have noticed that the operation to write the data to the system database tends to be quite slow. I don't think that there's a way to speed that up.

SimpleMobileTools / Simple-SMS-Messenger

SMS import does not work #646