Add MaxDocumentLength and custom UserAgent support

badoux / goscraper

Golang pkg to quickly return a preview of a webpage (title/description/images)

MIT License

108 stars 38 forks source link

Add MaxDocumentLength and custom UserAgent support #13

Open nightbloos opened 4 years ago

nightbloos commented 4 years ago

Due to the reason that sometimes URLs can be to some pages/files that are a pretty big one - we want to be able to abort the reading body for those pages.