kensanata / mastodon-archive

Archive your statuses, favorites and media using the Mastodon API (i.e. login required)
https://alexschroeder.ch/software/Mastodon_Archive
GNU General Public License v3.0
362 stars 33 forks source link

Archive broken after interrupt #46

Closed lapineige closed 5 years ago

lapineige commented 5 years ago

My archive seems to be broken. I launched a mastodon-archive command by mistake, and interrupted it during the Loading archive step. Now running any mastodon-archive command (including archiving) raise an error:

File "/home/myaccount/.local/bin/mastodon-archive", line 10, in <module>
   sys.exit(main())
File "/home/myaccount/.local/lib/python3.7/site-packages/mastodon_archive/__init__.py", line 259, in main
   args.command(args)
File "/home/myaccount/.local/lib/python3.7/site-packages/mastodon_archive/expire.py", line 71, in expire
   data = core.load(status_file, required = True)
File "/home/myaccount/.local/lib/python3.7/site-packages/mastodon_archive/core.py", line 190, in load
   return json.load(fp)
File "/usr/lib/python3.7/json/__init__.py", line 296, in load
   parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
   return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
   obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode
   obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1870349 column 30 (char 89941590)

How can I fix this ?

kensanata commented 5 years ago

It seems that you interrupted it while writing an archive? How does the file look to you at line 1870349? I'm suspecting that it just ends abruptly and that in effect you have data loss. In that case, I think your only option is to restore your archive from backup. Do you have a file that ends in .json~ that is larger than your archive? If so, you can try renaming it. Before saving the archive, we rename the old one and append a tilde just in case.

lapineige commented 5 years ago

How does the file look to you at line 1870349?

Well I don't know which file it is, they are many of them. The json ?

Do you have a file that ends in .json~ that is larger than your archive?

Well, current archive is 83Mo, json~ one is 500Mo… I guess that normal, as I interrupted the writing shortly after it started ?

Before saving the archive, we rename the old one and append a tilde just in case.

That's pretty clever ! I actually have a backup, but 1 month old, so that will save me a lot of time ! Thanks a lot :)