search

gptscript-ai / knowledge

Knowledge for GPTScript
https://gptscript-ai.github.io/knowledge/
Apache License 2.0
29 stars 14 forks source link

feat: per-file ingestion progress using structured (json) logs #116

Closed iwilltry42 closed 1 month ago

iwilltry42 commented 2 months ago

Example log structure (two lines):

[{
  "action": "ingestion",
  "file": "foo.pdf",
  "level": "INFO",
  "stage": "documentloader",
  "status": "progress",
  "step": "reading"
},
{
  "action": "ingestion",
  "file": "foo.pdf",
  "level": "INFO",
  "stage": "textsplitter",
  "status": "progress",
  "step": "splitting",
  "currentDoc": 5,
  "totalDocs": 10
}]