Closed atgreen closed 5 months ago
You can serialize them to json using node.model_dump()
(uses pydantic under the hood). I will add a section to the docs on it.
Added section to README + docs
Is it really well-formed JSON?
Should... {'variant': {'text'}, ..really be.. {'variant': 'text',
?
To get valid json you can run parsed_basic_doc.json()
. A node can be composed of text
, table
or image
elements so the variant is a set (or list in json).
What is the correct syntax for outputting in json? I'm trying to scrape a pdf table.
>>> parsed_basic_doc.json = parser.parse(basic_doc_path)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.local/lib/python3.11/site-packages/pydantic/main.py", line 839, in __setattr__
raise ValueError(f'"{self.__class__.__name__}" object has no field "{name}"')
ValueError: "ParsedDocument" object has no field "json"
In python you first assign the result to a variable and then call the json property.
from openparse import processing, DocumentParser
semantic_pipeline = processing.SemanticIngestionPipeline(
openai_api_key=OPEN_AI_KEY,
model="text-embedding-3-large",
min_tokens=64,
max_tokens=1024,
)
parser = DocumentParser(
processing_pipeline=semantic_pipeline,
)
parsed_content = parser.parse(basic_doc_path)
parsed_content.json()
Probably user error... but when I run your sample program in regular python, the printed nodes don't look like structured node output I expect. Instead I get...
etc etc
I was expecting the json-looking output from the documentation. What am I looking at?