Adapt code to upstream XML changes

openzim / sotoki

StackExchange websites to ZIM scraper

GNU General Public License v3.0

217 stars 25 forks source link

Closed benoit74 closed 5 months ago

benoit74 commented 5 months ago

Fix #305

most probably due to upstream changes, the XML files found in dumps have changed a bit

re-encode all XML files from UTF-16-LE (
- we could have used a lib to detect upstream XML encoding (e.g. chardet) but I considered these changes are too rare to make it worth it
ignore deleted posts and answers (looks like they were not in the dump before?)
adjust magic number used to manipulate XML entries in file preparation

rgaudin commented 5 months ago

oops, I've merged the fixups ! 🤦‍♂️

benoit74 commented 5 months ago

You've also forgot to rebase before merging. Never mind, I will rewrite the history.

benoit74 commented 5 months ago

Now everything is aligned