rmax scrapy-redis issues

rmax / scrapy-redis

Redis-based components for Scrapy.

http://scrapy-redis.readthedocs.io

MIT License

5.54k stars 1.59k forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

add ')' to spiders.py

#202 AYiXi closed 3 years ago
0
Refactor max_idle_time as an attribute

#201 laggardkernel closed 2 years ago
4
Fix RedisStatsCollector._get_key()

#200 laggardkernel closed 3 years ago
1
doc: fix redis-py dependency version

#199 tsonglew closed 3 years ago
0
how can i get sorted url in scrapy_redis zset? and how can spider get url by scrapy_redis's zset？

#198 FSNH closed 3 years ago
2
redis stats

#196 nieweiming closed 3 years ago
9
redis stats

#195 nieweiming closed 3 years ago
0
Queues defined in queue.py not seem to be used anywhere?

#194 abhinavkulkarni closed 3 years ago
0
Added maximum idle waiting time MAX_IDLE_TIME_BEFORE_CLOSE.

#193 nieweiming closed 3 years ago
3
WRONGTYPE Operation against a key holding the wrong kind of value

#192 JQ-K closed 3 years ago
2
style: correction of wrongly written characters.

#191 Pactortester closed 3 years ago
0
Release new version 0.7.1

#190 Germey closed 3 years ago
1
Release/0.7.1rc1

#189 Germey closed 3 years ago
1
Pre-Release of 0.7.1

#188 Germey closed 3 years ago
2
Fix datetime issue of redis 3.x

#187 Germey closed 3 years ago
5
Add stats extension based on Redis

#186 Germey closed 3 years ago
1
Crawl urls not from corresponding redis key

#185 nghuyong closed 3 years ago
0
How do I get two different crawlers to use one same dupefilter set

#184 MonkeyOnFire closed 3 years ago
2
Please release a new version based on master branch

#183 Germey closed 2 years ago
12
When does the crawler stop

#182 forgeries opened 3 years ago
4
Logs report a warning: Spider.make_requests_from_url method is deprecated...

#181 LittleHello opened 3 years ago
2
macOS big sur python3.8.2, __init__() got an unexpected keyword argument 'encoding'

#180 HOWIE-CH closed 2 years ago
1
scrapy性能问题

#179 ycq0125 closed 3 years ago
2
AttributeError: Can't pickle local object 'Crawler.__init__.<locals>.<lambda>'

#178 ycq0125 closed 4 years ago
5
If a url fails, is it added to dupefilter? Or just is given up.

#177 Vincent-Xin closed 3 years ago
4
init batch tasks

#176 qshine closed 4 years ago
0
You ever used python 3.8 run this spider connection same redis server.

#175 chenxuefeng346 closed 3 years ago
2
Item pipelines are slowed down

#174 Rossil2012 closed 1 year ago
2
create test

#173 dex0423 closed 4 years ago
4
feature: support priority start_urls

#172 qshine closed 4 years ago
2
Looking for co-maintainers!

#171 rmax opened 4 years ago
9
scrapy_redis/picklecompat builtins.ValueError: unsupported pickle protocol: 5

#170 StudyExchange closed 3 years ago
3
use pipeline when read redis list queue

#169 qshine closed 4 years ago
2
Spider only dequeued half url enqueued.

#168 StillHanWind closed 4 years ago
0
Should we auto reconnect to redis?

#167 botzill opened 4 years ago
2
support to configure different redis-server db by adding REDIS_DB

#166 featureoverload closed 4 years ago
1
why doesn't my spider_name:request has data in redis?

#165 hunger-Jack closed 3 years ago
2
fix(example): update dmoz spider domain

#164 tkrugg closed 4 years ago
1
scrapy-redis监听的redis_key是在需要哪个db里面lpush

#163 2424162 closed 3 years ago
6
RFPDupeFilter:: It seems this doesn't work

#162 forgeries closed 2 years ago
3
Some redis key questions about queue with requests、dupefilter and start_urls.

#161 Zerorigin closed 4 years ago
2
Help

#160 BB-9E closed 4 years ago
4
read start_urls use redis pipeline

#159 qshine closed 4 years ago
0
捕获http 404、402、503..

#158 wangtaihong closed 3 years ago
4
use pipeline from start_urls

#157 qshine closed 4 years ago
0
define redis record as url +parameters and access parameters in parse method

#156 ni-pixel opened 5 years ago
0
Multi-level url deduplication problem 多级url去重的问题

#155 CWF-999 closed 2 years ago
4
What is the logic about Base class key

#154 ZenRay closed 5 years ago
0
Scrapy-redis automatically helps create a xxx:request request queue in both libraries of the redis database

#153 hewm closed 1 year ago
0
The distributed scrappy crawler idling after requesting for a while and still having a list of requests in redis

#152 hewm closed 1 year ago
0

Previous Next