Because of #19 , the type hint for Spider.crawl_result broke, and it was temporarily replaced with Dict[str, Dict[str, Any]].
This should be fixed to actually reflect the contents of crawl_result, which has the following format:
crawl_result = {
"url1":{
"urls":["some url", "some other url", ...],
"body": "the html of the page"
},
"url2":{
"urls":["some url", "some other url", ...],
"body": "the html of the page"
},
}
Where body is only present if the include_body argument is set to True, and as such might not always be present.
See #19 for previous discussions about this.
You can verify the type hint is working if the mypy checks pass.
Because of #19 , the type hint for
Spider.crawl_result
broke, and it was temporarily replaced withDict[str, Dict[str, Any]]
. This should be fixed to actually reflect the contents ofcrawl_result
, which has the following format:Where
body
is only present if theinclude_body
argument is set toTrue
, and as such might not always be present. See #19 for previous discussions about this. You can verify the type hint is working if the mypy checks pass.