Closed hawktang closed 1 year ago
def load_data(
self, file: Path, metadata: Optional[Dict] = None
) -> List[Document]:
....
if metadata is None:
metadata = {}
metadata.update({"page_label": page_label, "file_name": file.name})
Subject: [PATCH] fix metadata error in pdf reader
---
Index: llama_index/readers/file/docs_reader.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/llama_index/readers/file/docs_reader.py b/llama_index/readers/file/docs_reader.py
--- a/llama_index/readers/file/docs_reader.py (revision d394ffd5b57b976192f002f52fc9315401b4aa09)
+++ b/llama_index/readers/file/docs_reader.py (revision 4109d5165fc79e68014702cbad91b1692fe4ad79)
@@ -14,7 +14,7 @@
"""PDF parser."""
def load_data(
- self, file: Path, extra_info: Optional[Dict] = None
+ self, file: Path, metadata: Optional[Dict] = None
) -> List[Document]:
"""Parse file."""
try:
@@ -36,10 +36,9 @@
# Extract the text from the page
page_text = pdf.pages[page].extract_text()
page_label = pdf.page_labels[page]
-
- metadata = {"page_label": page_label, "file_name": file.name}
- if extra_info is not None:
- metadata.update(extra_info)
+ if metadata is None:
+ metadata = {}
+ metadata.update({"page_label": page_label, "file_name": file.name})
docs.append(Document(text=page_text, metadata=metadata))
return docs
I saw you have fixed the issue in the repo. However, better give metadata even without explicit file_metadata function exist, right?
@hawktang not sure what you mean here. Is there an issue with the fix I made the other day? https://github.com/jerryjliu/llama_index/blob/d394ffd5b57b976192f002f52fc9315401b4aa09/llama_index/readers/file/docs_reader.py#L40
After your fix it is what I want now, thank you for the fix.
When will the fix publish with pip ;-)
Bug Description
[Bug]: PDFReader ignore extra_info/metadata from the file_metadata function
Version
0.6.35
Steps to Reproduce