-
I was wondering about the .. purpose(s) of rave.
I can see it's primary objective is to allow very easy development of modular (amd/cjs/es6) web apps where you don't have to worry about configuring a…
-
I am crawling a web that when call a image it is created on the fly, it returns 201 (create) in response.status, but this is not saved because response.status != 200.
https://github.com/scrapy/scrapy…
-
When using the appcache [edit: or older versions of IE], Iron Router converts a route like `/one` to `/#/one`.
Using `#!` would support the AJAX search engine crawling specification: https://develope…
-
Will this project be remade to use the information from opentransportdata.swiss instead of crawling the Sbb page? Would be a lot faster and more independent from the Sbb page.
-
I want to search files stored in a nas that's accessible by web or command line. Is there an api in spyglass to do the same job?
-
First, I want to say thank you. The icrawler does have me a lot.
I have read the source code, but didn't find a way to specify the interval for crawling
a website. Is it possible to specify the in…
-
for example:
```
crawler.crawl({
url: "http://localhost:8080/locations/",
selector: ".main-content"
```
would only follow the links found inside `.main-content`
this way i don't have to keep cr…
duggi updated
8 years ago
-
when crawling the files group, they are sorted alphabetically, moving index.js file at which ever place. I would be helpful to move the index.js (if found) to the top of each group, so its the first t…
acatl updated
9 years ago
-
## What happened?
Crawler never finishes when crawling `http://www.ohioattorneygeneral.gov/`.
## What should have happened?
There should be a maxRedirects property, which limits the amount of red…
-
# Value Added
Consolidates results for different subdomains against the same overall domain. Enables crawling of links that are on different subdomains
# Description
Currently the crawl servi…