Closed petermr closed 9 years ago
Actually maybe because I removed attribute: content
. But null
needs trapping.
the problem is that this:
"fulltext_html": {
"selector": "/",
"download": {
"rename": "fulltext.html"
}
},
Doesn't make sense. download
can only work when the captured element is a URL: it will download the URL. It can't just download arbitrary elements.
However, we should definitely be raising an informative error when this happens, so the user knows what they've done wrong.
thresher now notices what went wrong (https://github.com/ContentMine/thresher/commit/ac1f6ab94dbd0839e9b3868fd39f37760e99a481), and quicksrape reports what proportion of elements were successfully captured for each URL (f44d8dcc731f97195d139725baa207c8ebf63e00).
Cause unknown. Possibly bad scraper (attempting to download self for HTML, possibly following a closed HTML button).
scraper: