cash
commented
5 years ago - Radioaustralia does not have its own website anymore but has been folded into abc.net.au. It also dropped a few languages which I've removed. They also do not have a sitemap anymore so had to crawl links. Lastly, their html is a little difficult to scrape using just regular expressions, but I didn't want to add any new dependencies.
- Radio Free Asia now uses https and the redirects were breaking the crawler
- PNG Scriptures moved to a new hostname
sffc
brawer