results = doc.select("table p > a")
for result in results:
...
Unfortunately, if the webpage changes and doc.select returns an empty list we have no way of knowing that the scraper is now broken. The correct solution is probably to go back and change to something like
results = doc.select("table p > a")
if not results:
raise AssertionError("No report links found for %s" % url)
for result in results:
...
One common scraper pattern is
Unfortunately, if the webpage changes and
doc.select
returns an empty list we have no way of knowing that the scraper is now broken. The correct solution is probably to go back and change to something like