dosubot[bot]
commented
1 month ago Hey @drunkwcodes! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!
To resolve the "ModuleNotFoundError: No module named 'pandas.compat'" error when using SimpleDirectoryReader with MarkdownReader, you can configure SimpleDirectoryReader to exclude readers that depend on pandas. Here's how you can do it:
from llama_index.core.readers.file.base import SimpleDirectoryReader
from llama_index.readers.file.markdown import MarkdownReader
# Define a custom file_extractor that excludes Pandas readers
custom_file_extractor = {
".md": MarkdownReader,
# Add other readers as needed, excluding those that require pandas
}
# Initialize SimpleDirectoryReader with the custom file_extractor
reader = SimpleDirectoryReader(
input_files=["/home/drunkwcodes/projects/myfoam/kb/package_manager/pdm.md"],
file_extractor=custom_file_extractor
)
# Use the reader to read files
documents = reader.load_data()This configuration ensures that SimpleDirectoryReader uses MarkdownReader for .md files without requiring pandas [1].
To continue talking to Dosu, mention @dosu.
drunkwcodes
Bug Description
I was trying to do a simple markdown RAG:
But the result is still in vain:
ModuleNotFoundError: No module named 'pandas.compat'
The c extensions is not built by installing from pypi.
Please purge the pandas dependency for the majority of people who want plain text reading.
Version
0.11.14
Steps to Reproduce
pdm add these:
"llama-index>=0.11.14", "llama-index-embeddings-huggingface>=0.3.1", "llama-index-readers-pdf-marker>=0.2.0", "llama-parse>=0.5.6", "nest-asyncio>=1.6.0", "marker-pdf>=0.2.17", "llama-index-readers-file>=0.2.2", "pandas[excel]>=2.2.3",
and run this script:
Relevant Logs/Tracbacks
No response