issues
search
scrapinghub
/
frontera
A scalable frontier for web crawlers
BSD 3-Clause "New" or "Revised" License
1.29k
stars
216
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Kafka-python update and params change
#315
sibiryakov
closed
6 years ago
0
DBW components teardown
#314
sibiryakov
closed
6 years ago
0
LRU cache for HBaseState
#313
sibiryakov
closed
6 years ago
0
Stats log
#312
sibiryakov
closed
6 years ago
0
auto_start not defined? -- sorry, found it!
#311
MitchellChu
closed
6 years ago
0
HBaseState flush when frontier stop
#310
clarksun
opened
6 years ago
2
update BCPerHostLimit in cluster example
#309
clarksun
closed
6 years ago
0
Update db.py
#308
clarksun
opened
6 years ago
3
maybe miss b?
#307
clarksun
closed
6 years ago
1
Update hbase.py
#306
clarksun
closed
6 years ago
1
Do not remove existing loggers on configuration
#305
sibiryakov
closed
6 years ago
1
Improving errors logging in async offsets fetcher
#304
sibiryakov
closed
6 years ago
0
Smaller kafka client poll intervals
#303
sibiryakov
closed
6 years ago
1
Add message type logging and time for batch
#302
sibiryakov
closed
6 years ago
1
HBase states table option
#301
sibiryakov
closed
6 years ago
1
How to yield splash request when using seed file
#300
MuhammadRahman-awin
opened
6 years ago
9
hdr and redir persistence + cityhash64 + graceful shutdown fix + kafka SSL
#299
sibiryakov
closed
6 years ago
1
Duplicate Entries
#298
ijharulislam
opened
6 years ago
7
when i used json,something wrong happend
#297
EchoShoot
closed
11 months ago
4
Add max_queue_size to overused buffer to stop queue request when the …
#296
isra17
closed
6 years ago
3
Set overused factor so it actually trigger when a slot is full.
#295
isra17
closed
6 years ago
1
Help Wanted. Can not run example/scrapy_recording, NameError: global name 'graphs' is not defined
#294
imfht
opened
6 years ago
0
Frontera Request/Response decoding/encoding doesnt work between python 2/3 versions
#293
whalebot-helmsman
opened
6 years ago
1
Spider being cancelled (stalled)
#292
ijharulislam
opened
7 years ago
0
Only half of spiders are crawling
#291
colmex
opened
7 years ago
0
Working example project?
#290
colmex
closed
7 years ago
4
`AssertionError` can't find cause for it
#289
colmex
closed
7 years ago
9
Improve HBase states cache with LRU
#288
vshlapakov
closed
6 years ago
1
SSL support for Kafka message bus
#287
vshlapakov
closed
6 years ago
1
Use CityHash for FingerprintPartitioner
#286
vshlapakov
closed
6 years ago
1
no module named recording while trying to record the scrapy crawl
#285
SharmaSushant19
opened
7 years ago
0
Ensure partitioning is consistent accross message bus and queue
#284
isra17
opened
7 years ago
1
Fix consistent partitionning
#283
isra17
closed
7 years ago
0
Windows support for Frontera
#282
11rohans
closed
5 years ago
6
Mark partition as busy when a new batch is sent to it
#281
isra17
opened
7 years ago
5
Temporary fix of HBaseQueue.get performance
#280
sibiryakov
closed
7 years ago
1
Fix hostname local fingerprint from collisions
#279
sibiryakov
closed
7 years ago
1
HBase and splash
#278
International
opened
7 years ago
2
Fix hostname local fingerprint from collisions
#277
sibiryakov
closed
7 years ago
4
Fix of requests propagation to Scheduler from middlewares and engine
#276
sibiryakov
opened
7 years ago
7
Unify backend logging
#275
ZipFile
closed
2 years ago
4
Commit database cleared content
#274
isra17
closed
7 years ago
2
Revert "Fixed scheduler process_spider_output() to yield requests"
#273
voith
closed
7 years ago
3
Fix error raising in msgpack and json decoders
#272
ZipFile
closed
7 years ago
3
Update frontier-backends.rst
#271
xsren
closed
7 years ago
1
Update frontier-backends.rst
#270
xsren
closed
7 years ago
2
Gracefully restarting frontera with SQL backend
#269
ShT3ch
opened
7 years ago
4
Msgpack warning fix for None objects and Offsets-related error fix
#268
sibiryakov
closed
7 years ago
1
Msgpack warning fix for None objects
#267
sibiryakov
closed
7 years ago
0
Strategy worker error
#266
kazuar
opened
7 years ago
9
Previous
Next