Excpetion case no robot file

deibit / cansina

Web Content Discovery Tool

GNU General Public License v3.0

888 stars 237 forks source link

Excpetion case no robot file #11

Closed XbandoleroX closed 7 years ago

XbandoleroX commented 7 years ago

Using robots.txt as payload
Traceback (most recent call last):
  File "cansina.py", line 215, in <module>
    print("Reaped %s entries" % (len(robots_content)))
TypeError: object of type 'NoneType' has no len()

deibit commented 7 years ago

Fixed in #13