internetarchive / dweb-mirror

Offline Internet Archive project
https://www-dweb-mirror.dev.archive.org/
GNU Affero General Public License v3.0
261 stars 27 forks source link

crawl fails to get meta.xml #319

Closed mitra42 closed 4 years ago

mitra42 commented 4 years ago

crawl all opens commute_meta.xml.part but fails silently somewhere before it saves the file.

See https://github.com/internetarchive/dweb-mirror/issues/140 which will be wrong till this fixed

mitra42 commented 4 years ago

Fixed, was a problem with dweb-cors which passed on a content-encoding header: zip, DM couldnt decode this since it wasnt expecting zip.