shababo / plotreader

Use LLMs to generate and read plots. Combine them into a teacher-student pair to improve performance.
1 stars 0 forks source link

ingestion pipeline - capture the text and section structure of the paper into nodes #4

Closed shababo closed 1 month ago

shababo commented 1 month ago

First of three ingestion pipelines that will help us understand and coordinate information throughout a scientific paper. This one will use llmsherpa to extract the text and section structure of the paper. We've already validated this on our example paper, just need to refactor out into an ingestion pipeline.