Issue warning instead of halting if encountering an error when loading files during ingest

amaiya / onprem

A tool for running on-premises large language models with non-public data

https://amaiya.github.io/onprem

Apache License 2.0

684 stars 32 forks source link

Closed amaiya closed 3 months ago

amaiya commented 3 months ago

Issue warning instead of halting if encountering an error when loading files during ingest. Show files skipped due to errors.

Example:

WARNING:OnPrem.LLM-ingest:Skipping /home/amaiya/errordata/problem_file.pdf due to error: Error loading /home/amaiya/errordata/problem_file.pdf