Some nested blocks (eg. <table> inside <p>) cause the paragraph break parser to fail. Although this nesting is not allowed in the spec, it has been seen in real pages.
Traceback (most recent call last):
File "populate_article_db.py", line 38, in <module>
main()
File "populate_article_db.py", line 34, in main
use_local=args.local)
File "/Users/jrobinson/Projects/misinformation/misinformation-crawler/misinformation/warc/warc_parser.py", line 131, in process_webpages
response = response_from_warc(warc_data)
File "/Users/jrobinson/Projects/misinformation/misinformation-crawler/misinformation/extractors/extract_article.py", line 41, in extract_article
default_readability_article = simple_json_from_html_string(page_html, content_digests, node_indexes, use_readability=False)
File "/Users/jrobinson/Projects/misinformation/misinformation-crawler/ReadabiliPy/readabilipy/simple_json.py", line 34, in simple_json_from_html_string
"content": str(simple_tree_from_html_string(html))
File "/Users/jrobinson/Projects/misinformation/misinformation-crawler/ReadabiliPy/readabilipy/simple_tree.py", line 42, in simple_tree_from_html_string
insert_paragraph_breaks(soup)
File "/Users/jrobinson/Projects/misinformation/misinformation-crawler/ReadabiliPy/readabilipy/simplifiers/html.py", line 198, in insert_paragraph_breaks
if parent_element.name == "p":
AttributeError: 'NoneType' object has no attribute 'name'
Some nested blocks (eg.
<table>inside<p>) cause the paragraph break parser to fail. Although this nesting is not allowed in the spec, it has been seen in real pages.