If page too large for screenshot crawl crashes

hoarder-app / hoarder

A self-hostable bookmark-everything app (links, notes and images) with AI-based automatic tagging and full text search

https://hoarder.app

GNU Affero General Public License v3.0

6.48k stars 235 forks source link

If page too large for screenshot crawl crashes #583

Open Explosion-Scratch opened 1 month ago

Explosion-Scratch commented 1 month ago

Describe the Bug

If a webpage is too large to screenshot the crawl crashes, and doesn't save cached or an archive.

Steps to Reproduce

Crawl this page: https://gwern.net/embryo-selection

Expected Behaviour

More a feature request but I'd love if it could just save each part that succeeded (cached content, webpage archive) instead of throwing away the other things

Screenshots or Additional Context

No response

Device Details

No response

Exact Hoarder Version

v0.18.0

MohamedBassem commented 1 month ago

Just for my understanding, do you have full page screenshot enabled?

Explosion-Scratch commented 1 month ago

@MohamedBassem Yep