scrapinghub frontera issues

scrapinghub / frontera

A scalable frontier for web crawlers

BSD 3-Clause "New" or "Revised" License

1.29k stars 217 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Remove logs

#413 adityaraj-28 closed 10 months ago
0
Tweak

#412 adityaraj-28 closed 10 months ago
0
Inconsistent data types within library

#411 adityaraj-28 opened 11 months ago
0
how can I know it works when I use it with scrapy?

#410 vidyli opened 3 years ago
1
Project Status?

#409 psdon opened 3 years ago
8
How To setup a cluster

#408 gurpartap1986 opened 3 years ago
1
Message Decode Error

#407 ab-bh opened 3 years ago
0
Usage of Kafka instead of ZeroMQ

#406 ab-bh closed 3 years ago
0
Unable to install options with frontera

#405 ab-bh closed 3 years ago
0
The `KeyError` throw when running to to_fetch in StateContext class: b'fingerprint'

#404 yujiaao opened 3 years ago
0
Fix check and create tables with is_clear flag

#403 dorellang closed 3 years ago
0
Linking custom strategy

#402 b0tf4th3r closed 3 years ago
1
KeyError [b'frontier'] on Request Creation from Spider

#401 dkipping opened 3 years ago
3
ModuleNotFoundError: No module named 'frontera.contrib.scrapy.middlewares.recording'

#400 gesof opened 4 years ago
1
Updated cluster-setup.rst

#399 gesof opened 4 years ago
0
[BUG] Clear content and drop tables options are not working properly

#398 Prometheus3375 opened 4 years ago
0
"Can't identify DB worker type" running DB worker

#397 gesof closed 4 years ago
4
Metadata is not saved to db in single process mode

#396 Prometheus3375 opened 4 years ago
2
Error processing robots.txt - no body

#395 Prometheus3375 closed 4 years ago
2
Could not decode message - error unpackb() got an unexpected keyword argument 'encoding'

#394 incognitozen opened 4 years ago
1
HBaseQueue losing urls when using BC_MAX_REQUESTS_PER_HOST

#393 a-shkarupin closed 4 years ago
1
Update/remove documentation on the 'revisiting backend'

#392 anjackson opened 4 years ago
2
fixed incorrect import module

#391 EkkoG closed 3 years ago
2
In which place i can add my url prioritizing strategy in frontera,. i couldn't find any clear documentation related to that.

#390 arun477 closed 4 years ago
1
add assertion error description for easier debugging

#389 a-shkarupin closed 4 years ago
1
Explicit version for kafka session timeout

#388 a-shkarupin opened 4 years ago
1
Enable test_scrapy_spider() test as it seems to be working now

#387 suzil closed 4 years ago
1
Add a test for generating graph visualizations

#386 suzil closed 4 years ago
2
Replace pydot with graphviz

#385 suzil closed 3 years ago
0
Add "make run" command

#384 suzil closed 4 years ago
2
Move tox to test requirements

#383 suzil closed 4 years ago
2
Remove manual install of docker-compose in .travis.yml

#382 suzil closed 4 years ago
0
Add badges

#381 suzil closed 4 years ago
1
Use absolute imports

#380 suzil closed 4 years ago
4
Bump to Python 3.6 + 3.7

#379 suzil closed 4 years ago
4
ERROR:sqlalchemy.queue:A transaction is already begun. Use subtransactions=True to allow subtransactions.

#378 yevgenpapernyk opened 4 years ago
0
SQLAlchemy: AttributeError: _Connection__connection

#377 yevgenpapernyk opened 4 years ago
1
Module 'frontera.contrib.backends.memory' doesn't define any object named 'FIFO'

#376 Rockyzsu opened 5 years ago
3
Update models.py

#375 ghost closed 5 years ago
1
Decode spider logs self.body return none error

#374 ghost closed 5 years ago
3
db worker and strategic worker crashed unexpectedly

#373 ghost closed 5 years ago
2
Can seed specify crawler run?

#372 z10z10b10 closed 5 years ago
1
ModuleNotFoundError: No module named 'frontera.contrib.scrapy.middlewares.seeds'

#371 ghost opened 5 years ago
12
Does Apache Kafka need Apache Zookeeper to run Frontera?

#370 ghost closed 5 years ago
1
Is the project still in progress？

#369 kingkap closed 5 years ago
2
Failed building wheel for cityhash

#368 ghost closed 5 years ago
0
pip install Failed with exit status 2

#367 EinAeffchen opened 5 years ago
2
Tests fix

#366 sibiryakov closed 5 years ago
0
Fixed DomainCache crash on init for Python 3.6

#365 sibiryakov closed 5 years ago
0
Add 3.6 support in setup.py

#364 lucywang000 closed 4 years ago
2