search

neuml / paperetl

📄 ⚙️ ETL processes for medical and scientific papers
Apache License 2.0
352 stars 27 forks source link

example data processing warning using google colab #54

Open amscosta opened 8 months ago

amscosta commented 8 months ago

Hello, The following warning is issued when processing one of the .xml from the example data: Processing: paperetl/file/data/0.xml /usr/local/lib/python3.10/dist-packages/paperetl/file/tei.py:35: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument features="xml" into the BeautifulSoup constructor. soup = BeautifulSoup(stream, "lxml")

Any clue how to avoid/correct that? Thanks a lot.

amscosta commented 8 months ago

I am using the colab notebook.

davidmezzetti commented 8 months ago

You can ignore it like this:

import warnings
from bs4 import XMLParsedAsHTMLWarning

warnings.filterwarnings("ignore", category=XMLParsedAsHTMLWarning)
amscosta2022 commented 8 months ago

Thanks. But "using an XML parser will be more reliable" the message says.

davidmezzetti commented 8 months ago

Feel free to fork this project and try. It doesn't work in the tests I've run.