Separate media archiving - Githubissues

bellingcat / cisticola

Coordinates scrapers and interfaces with database

15 stars 0 forks source link

Separate media archiving #19

Closed loganwilliams closed 2 years ago

loganwilliams commented 2 years ago

Each Scraper needs to be updated to:

add original URLs to the archived_urls dict even if media isn't being downloaded
implement archive_files if necessary to allow files to be downloaded later

Implemented in scrapers:

[x] TwitterScraper
[x] TelegramTelethonScraper
[ ] RumbleScraper
[ ] OdyseeScraper
[ ] InstagramScraper
[ ] GettrScraper
[ ] GabScraper
[ ] BitchuteScraper
[ ] VkontakteScraper
[ ] YoutubeScraper
[x] TelegramSnscrapeScraper

This shouldn't be merged after #17 is reviewed and merged.