issues
search
istresearch
/
scrapy-cluster
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
http://scrapy-cluster.readthedocs.io/
MIT License
1.18k
stars
323
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
ERROR: Unable to connect to Kafka in Pipeline due to attempt to connect already-connected SSLSocket!, raising exit flag.
#269
BeamoINT
opened
11 months ago
1
runspider: error: Unable to load 'link_spider.py': attempted relative import with no known parent package
#268
BeamoINT
closed
10 months ago
4
Upgrade the project to python 3.10
#267
borisjota
opened
1 year ago
4
Is this project no longer supported?
#266
flyingdev
closed
1 year ago
2
Update Dockerfile
#265
shivaabed1994
opened
2 years ago
0
fix missing package python-lxml
#264
itisrasoul
closed
2 years ago
0
lxml dependency is broken for python3.6 base image
#263
rafaharo
opened
2 years ago
0
Updated Names and statement for localsettings.py
#262
abhalawat
opened
2 years ago
0
Crawler image - Python dependency fix
#261
igortelheiro
closed
2 years ago
0
ui exception No connection adapters were found
#260
new-wxw
opened
2 years ago
0
Crawler unhandled exceptions not logged
#259
getorca
closed
3 years ago
1
scrapy-cluster don't scroll the entire pages
#258
RochdiBoudokhane
closed
3 years ago
7
Fix online dmoz test
#257
madisonb
closed
3 years ago
0
Update quickstart.rst
#256
oprog
closed
3 years ago
0
Stats keys flooding redis + Crawl resilience + retry strategy
#255
benjaminelkrieff
closed
3 years ago
5
Incremented fail stats
#254
mingxuan1
closed
3 years ago
0
Upgrading the ELK stack
#253
4OH4
opened
3 years ago
2
TypeError: can't pickle thread.lock objects
#252
benjaminelkrieff
opened
3 years ago
2
Jobs disappearing: is there a way to monitor crawls ?
#251
benjaminelkrieff
closed
3 years ago
4
Scrape delay
#250
tluyben
closed
3 years ago
2
docs: fix grammatical errors in docker.rst
#249
mslowikowski
closed
3 years ago
1
Bump jinja2 from 2.10 to 2.10.1 in /rest
#248
dependabot[bot]
closed
3 years ago
1
Bump requests from 2.18.4 to 2.20.0 in /rest
#247
dependabot[bot]
closed
3 years ago
1
Bump werkzeug from 0.14.1 to 0.15.3 in /rest
#246
dependabot[bot]
closed
3 years ago
1
Bump requests from 2.18.4 to 2.20.0 in /crawler
#245
dependabot[bot]
closed
3 years ago
1
Bump requests from 2.18.4 to 2.20.0 in /kafka-monitor
#244
dependabot[bot]
closed
3 years ago
1
Bump twisted from 18.4.0 to 20.3.0 in /crawler
#243
dependabot[bot]
closed
3 years ago
1
Dependency Resolution Fixes
#242
madisonb
closed
3 years ago
0
maxdepth can not large than 2
#241
anthony9981
closed
3 years ago
5
Bump cryptography from 1.8.1 to 3.2
#240
dependabot[bot]
closed
3 years ago
1
Bump cryptography from 1.8.1 to 3.2 in /crawler
#239
dependabot[bot]
closed
3 years ago
1
Update Releases, Docker compose Issues
#238
MrMoon
closed
3 years ago
2
Ansible deprecated sudo
#237
j3k00
closed
3 years ago
1
Add missing ELK restart instructions to docker-compose file
#236
Shique
closed
3 years ago
1
Future of the project
#235
demisx
opened
4 years ago
9
Fixed an error with MarkupSafe installation
#234
Shique
closed
4 years ago
2
Fixed an error with MarkupSafe installation
#233
Shique
closed
4 years ago
0
Any suggestion on the best way to get the crawled data out to a file?
#232
ahoquegh
closed
4 years ago
0
How to submit a POST request through crawler REST API
#231
YanzhongSu
closed
4 years ago
2
Python 3
#230
deed02392
opened
4 years ago
6
Why encoding byted body
#229
YanzhongSu
closed
4 years ago
2
Bump werkzeug from 0.12.1 to 0.15.3
#228
dependabot[bot]
closed
4 years ago
1
Can't up cluster from master branch
#227
MasterSergius
closed
4 years ago
1
supervisord install error while vagrant up
#226
fatemeeeeeee
closed
4 years ago
2
A mistake in parameter deliver through two method!
#225
kevin-ZZZ
closed
4 years ago
4
online integration test error
#224
rawiafray
closed
4 years ago
2
Cannot turn off DEBUG level log
#223
YanzhongSu
closed
5 years ago
2
Coveralls fix
#222
madisonb
closed
5 years ago
1
`Demo.incoming` topic keeps piling up
#221
YanzhongSu
closed
5 years ago
6
Docker ERROR: Could not ping Zookeeper
#220
Shique
closed
5 years ago
6
Next