Search for documents in a domain through Search Engines (Google, Bing and Baidu). The objective is to extract metadata
GNU General Public License v3.0
186
stars
32
forks
source link
Error donwloading (redirect case) #8
Closed
alanEG closed 3 years ago
hello here you parse url to cut name file https://github.com/Josue87/MetaFinder/blob/ff1a16c12e86969cc167e0fa37102eada8bae343/metafinder/utils/file/download.py#L23-L24 but if the url is
https://www.domain.com/es/photopassbslegales/
There will be a problem Because there will be nothing to cut['https:', '', 'www.domain.com', 'es', 'photopassbslegales', '']
why that Because you are taking the URL from https://github.com/Josue87/MetaFinder/blob/ff1a16c12e86969cc167e0fa37102eada8bae343/metafinder/utils/file/download.py#L20 This url is not valid now, but when you send the request, the file will be automatically redirected. The response.url will be assigned to the address to which it was redirected, so you must take the URL for processing from response.url