mishushakov / llm-scraper

Turn any webpage into structured data using LLMs
MIT License
2.41k stars 147 forks source link

Standardised testing for package #23

Open Ademsk1 opened 6 months ago

Ademsk1 commented 6 months ago

We need to have some standardised tests for what this LLM can and cannot do, and make the relevant choices. This issue should be used as a discussion for what websites are best for testing this package.

  1. Auth websites
  2. Long/short HTML websites
  3. any others?