Create a DB of URLs that redirect elsewhere and the redirected URL actually is in the sitemap. This would avoid needlessly getting the same old URL multiple times to get the final URL when the old URL isn't found in the sitemap.
Do this around the line with finalURL = requests.get(linkBaseURL).url
Create a DB of URLs that redirect elsewhere and the redirected URL actually is in the sitemap. This would avoid needlessly getting the same old URL multiple times to get the final URL when the old URL isn't found in the sitemap. Do this around the line with
finalURL = requests.get(linkBaseURL).url