c4software / python-sitemap

Mini website crawler to make sitemap from a website.
GNU General Public License v3.0
366 stars 110 forks source link

Add link that caused 404 to the sitemap #13

Closed kevinburke closed 11 years ago

kevinburke commented 11 years ago

My french is not great but here goes

Quand il y a un "HTTP Error 404" je me demande quel hyperlie a cause le problem.

c4software commented 11 years ago

Hi,

I will add in the next version, the list of url who are make 404 during the crawl (with the --report)

c4software commented 11 years ago

Added in the last commit 385035d58f316f7227ffe0e66127dd4537f10018 It was what you imagined?

c4software commented 11 years ago

python main.py --config config.json --report Number of found URL : 11 Number of link crawled : 11 Number of link exclude : 0 Nb Code HTTP 200 : 9 Nb Code HTTP 404 : 2 Uri with status 404: