asielen
commented
4 years ago Created a new media type: clipping Split out the compare_file function to allow different logic by file type. (PDFs don't compare easily, the hash is not stable)
New dependency, pathvalidate
New function: get_newspaper_clipping.
- Takes the session and a url.
- Downloads a file with the filename: newspapers_com--clippingtitle--clipplingID.pdf
- Stores them in a new clippling folder and updates the toml file for the record
New function: Check url note
- Checks the ancestry url for additional urls to check. For now it is just for Newspaper.com resources. It could be used for any external resources. (Download additional findagrave info for example)
Added a bit more error checking for missing files. I think this was happening when hitting ancestry with too many requests too quickly. So it will flag the file as unavailable which means it will look for the file again next time it runs.
Added functionality for pulling Newspapers.com pdf clippings