dosubot[bot]
commented
2 months ago To address the issue of ensuring UTF-8 encoding when persisting the PropertyGraphIndex, you can modify the persist() method to handle JSON serialization with the ensure_ascii=True parameter. Since the model_dump_json() method does not support this parameter directly, you can manually handle the JSON serialization.
Here's an example of how you can override the persist() method to achieve this:
import json
import fsspec
class CustomPropertyGraphIndex(PropertyGraphIndex):
def persist(self, persist_path: str, fs: Optional[fsspec.AbstractFileSystem] = None) -> None:
"""Persist the graph store to a file with ensure_ascii=True."""
if fs is None:
fs = fsspec.filesystem("file")
with fs.open(persist_path, "w") as f:
json.dump(self.graph.model_dump(), f, ensure_ascii=True)
# Usage
custom_index = CustomPropertyGraphIndex(
nodes=[], # Your list of nodes
llm=llm, # Your language model
kg_extractors=[custom_transform], # Your custom transform
property_graph_store=SimplePropertyGraphStore(),
use_async=True,
embed_kg_nodes=True,
show_progress=True
)
custom_index.persist("path/to/property_graph_store.json")In this example, the persist() method is overridden to use json.dump() with ensure_ascii=True, ensuring that non-ASCII characters are properly escaped.
This approach should solve the UTF-8 encoding issue for the property_graph_store.json file [1][2].
To continue talking to Dosu, mention @dosu.
912100012
Question Validation
Question
when I use the propertygraphindex to persist to disk，the other file such as “default__vector_store.json” saved as utf8 because it use the "json.dump(assic=True)" when vector_store.persist() is called.But the propertygraph.persist use the "model_dump_json()" when its persist() is called,the "model_dump_json()" do not have the assic=True paramaeter to cover the chinese to the format like “\u6235" So how can i solve this problem,only the property_graph_store.json file can not be saved as utf-8.