Josue87 / MetaFinder

Search for documents in a domain through Search Engines (Google, Bing and Baidu). The objective is to extract metadata
GNU General Public License v3.0
186 stars 32 forks source link

Error donwloading (redirect case) #8

Closed alanEG closed 3 years ago

alanEG commented 3 years ago

hello here you parse url to cut name file https://github.com/Josue87/MetaFinder/blob/ff1a16c12e86969cc167e0fa37102eada8bae343/metafinder/utils/file/download.py#L23-L24 but if the url is https://www.domain.com/es/photopassbslegales/ There will be a problem Because there will be nothing to cut ['https:', '', 'www.domain.com', 'es', 'photopassbslegales', ''] why that Because you are taking the URL from https://github.com/Josue87/MetaFinder/blob/ff1a16c12e86969cc167e0fa37102eada8bae343/metafinder/utils/file/download.py#L20 This url is not valid now, but when you send the request, the file will be automatically redirected. The response.url will be assigned to the address to which it was redirected, so you must take the URL for processing from response.url