search

c4software / python-sitemap

Mini website crawler to make sitemap from a website.
GNU General Public License v3.0
362 stars 110 forks source link

AttributeError: 'NoneType' object has no attribute 'geturl' #75

Open devopsenko opened 3 years ago

devopsenko commented 3 years ago

I got such error python3 main.py --domain https://domain.com --output sitemap.xml

Traceback (most recent call last): File "main.py", line 60, in crawl.run() File "/root/python-sitemap/crawler.py", line 127, in run self.__crawl(current_url) File "/root/python-sitemap/crawler.py", line 264, in __crawl final_url = response.geturl() AttributeError: 'NoneType' object has no attribute 'geturl'

c4software commented 3 years ago

Interesting what is your Python version please ?

devopsenko commented 3 years ago

Python 3.8.2

ChenKuanSun commented 3 years ago

It is showing on pdf file url exist. So content might be not parseable and response will be None.

Need add condition in final_url = response.geturl() like final_url = response.geturl() if response is not None else current_url

I create a PR #77

Pacifist2503 commented 2 years ago

It is showing on pdf file url exist. So content might be not parseable and response will be None.

Need add condition in final_url = response.geturl() like final_url = response.geturl() if response is not None else current_url

I create a PR #77

Please help me figure out in which file exactly which line to insert, what to exclude the specified error?