-
### I'm submitting a
[x] bug report
### ...about
[x] edit experience / UI
[x] admin experience UI
[x] DNN parts
### Current Behavior / Expected Behavior
The view configuration UI shows csht…
-
### Which package is this bug report for? If unsure which one to select, leave blank
@crawlee/core
### Issue description
The crawler, while running, will randomly crash Node. I tried using the expe…
-
## Description
We recently transitioned from Raindrop to Hoarder and using a custom script with the `hoarder-cli`, we were able to import my Raindrop collections and link them to the matching list on…
-
### Checklist
- [X] I am using **LTS** version of Node.js.
- [X] I am using the **latest v2 version** of VuePress and VuePress official plugins.
- [X] I am using the **latest v2 version** of vuepress…
-
Since crawler 0.11.0 (https://github.com/webrecorder/browsertrix-crawler/pull/362), the captured favicon is available in pages.jsonl
We could use that when a custom favicon is not provided instead of…
-
Recently came across the [Browsertrix Crawler](https://github.com/webrecorder/browsertrix-crawler) project which seems to be using Brave Browser for crawls. Some of its features include `Support for c…
-
@tnylea
- I just pushed the 'js' branch. it's not ready for PR yet but I want you to see this database setup to start thinking.
- You were right – JSON was the wrong direction.
- we'll be ab…
-
Is there a way to take a screenshot for a `process_spider_exception` error?
I can't figure out how to access the page object in that middleware.
-
A crawler might look something like this.
```
var Crawler = require('roach').job();
Crawler.extend({
// Your extra custom functions for parsing, etc.
custom: function(params, data, next){
//…
-
### Which package is this bug report for? If unsure which one to select, leave blank
None
### Issue description
Add lots of initial urls to the request queue
Once a url is scraped, add 2nd level s…