search

GDD-Nantes / LLM4SchemaOrg

0 stars 1 forks source link

Wayback Machine limitation #1

Closed mhoangvslev closed 9 months ago

mhoangvslev commented 1 year ago

15 requests / minutes + 5 minutes cooldown.

We need a way to retrieve the web data

mhoangvslev commented 9 months ago

Use CommonCrawl index to retrieve the website instead + exponential backoff