simplecrawler simplecrawler issues

simplecrawler / simplecrawler

Flexible event driven crawler for node.

BSD 2-Clause "Simplified" License

2.14k stars 354 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

docs: fix discoverResources signature in example

#498 bard opened 3 years ago
0
addDownloadCondition example?

#497 xeroxstar closed 3 years ago
1
Adding to the queue on "complete" callback doesn't work

#496 Ivanca opened 3 years ago
0
crawler.supportedMimeTypes not moving after first page

#495 ankurarora opened 3 years ago
0
addFetchCondition to get only text/html content type?

#494 msudol closed 4 years ago
2
Proxy for each request in the queue?

#493 alex-w0 opened 4 years ago
2
Generic error missing

#492 Ivanca opened 4 years ago
1
Bug fix: Remove fragment (to avoid bad 404s)

#491 Ivanca opened 4 years ago
0
Introduced fetch

#490 pcdeshmukh opened 4 years ago
0
Update jsdoc-to-markdown to the latest version 🚀

#489 greenkeeper[bot] opened 4 years ago
0
Fail to decode application/x-gzip

#488 FelixRe0 opened 4 years ago
0
An in-range update of mocha is breaking the build 🚨

#487 greenkeeper[bot] opened 4 years ago
2
Fix format in FS cache backend error constructor

#486 kbychkov closed 4 years ago
0
Fix format in FS cache backend error constructor (Take 3)

#485 Mr0grog closed 4 years ago
2
Fix format in FS cache backend error constructor (take 2)

#484 Mr0grog closed 4 years ago
1
Fix string substitution in FS cache backend error constructor

#483 Mr0grog closed 4 years ago
8
Update mocha to the latest version 🚀

#482 greenkeeper[bot] closed 4 years ago
2
docs: does this spider support "waiting for all the JS to finish" because actually crawling the resulting page?

#481 Pomax opened 4 years ago
2
How to await "fetchcomplete"?

#480 Pradyumna-medarametla opened 4 years ago
1
Which method to use avoid crawling URL's that end with .js /.css /.png/.jpg

#479 Pradyumna-medarametla closed 4 years ago
1
SQLite FetchQueue Implementation for Simplecrawler

#478 LeMoussel opened 4 years ago
0
An in-range update of eslint is breaking the build 🚨

#477 greenkeeper[bot] closed 4 years ago
2
Crawler stuck on url 'Exceeded maximum number of redirects'

#476 stijn-lcp opened 5 years ago
1
Adding fetchcondition to check broken links

#475 stijn-lcp closed 5 years ago
2
Request path contains unescaped characters

#474 guidodizi opened 5 years ago
0
Update iconv-lite to the latest version 🚀

#473 greenkeeper[bot] closed 5 years ago
0
Update eslint to the latest version 🚀

#472 greenkeeper[bot] closed 5 years ago
1
Cannot find site page

#471 kr-ilya closed 5 years ago
5
cache: send headers If-None-Match and If-Modified-Since

#470 deton closed 5 years ago
1
Update async to the latest version 🚀

#469 greenkeeper[bot] closed 5 years ago
0
Update jsdoc-to-markdown to the latest version 🚀

#468 greenkeeper[bot] closed 5 years ago
0
feature: validate outgoing request headers before sending them

#467 upugo-dev opened 5 years ago
0
Greenkeeper/mocha 6.1.1

#466 kbychkov closed 5 years ago
0
An in-range update of mocha is breaking the build 🚨

#465 greenkeeper[bot] closed 5 years ago
2
chore(package): update lockfile package-lock.json

#464 kbychkov closed 5 years ago
0
chore(package): update eslint to version 5.16.0

#463 kbychkov closed 5 years ago
0
chore: update travis installation script

#462 kbychkov closed 5 years ago
0
test: fix cookie expires

#461 kbychkov closed 5 years ago
0
An in-range update of eslint is breaking the build 🚨

#460 greenkeeper[bot] closed 5 years ago
2
fix jsdoc2md generated links

#459 kbychkov closed 5 years ago
0
Fix docs of events `crawlstart` and `discoverycomplete`

#458 siwinski closed 5 years ago
0
[README.md] Re-run `npm run docs`

#457 siwinski closed 5 years ago
3
Update dependencies to enable Greenkeeper 🌴

#456 greenkeeper[bot] closed 5 years ago
0
Update dependencies

#455 kbychkov closed 5 years ago
0
Rewrite deprecated use of `Buffer`

#454 kbychkov closed 5 years ago
2
chore: update badges

#453 kbychkov closed 5 years ago
1
Can't compile app because of simplecrawler

#452 bruceandroid opened 5 years ago
1
Fix: handle unwanted path char in chaching file path

#451 FranziskaP opened 5 years ago
0
fix: get rid of multiple `fetchQueueItem` execution for the same items

#450 kbychkov closed 5 years ago
0
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

#449 motiejuss opened 5 years ago
2