scrapinghub / frontera

A scalable frontier for web crawlers
BSD 3-Clause "New" or "Revised" License
1.29k stars 216 forks source link

Remove invalid keyword argument 'sleep' from kafka poll #345

Closed icapurro closed 6 years ago

icapurro commented 6 years ago

Removed the invalid keyword sleep from the poll call triggered after getting an exception about invalid metadata in the offset_fetcher.

Added a unit test to validate this particular scenario.

codecov[bot] commented 6 years ago

Codecov Report

Merging #345 into master will not change coverage. The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #345   +/-   ##
=======================================
  Coverage   59.95%   59.95%           
=======================================
  Files          78       78           
  Lines        6143     6143           
  Branches      809      809           
=======================================
  Hits         3683     3683           
  Misses       2284     2284           
  Partials      176      176
Impacted Files Coverage Δ
...ontera/contrib/messagebus/kafka/offsets_fetcher.py 14.89% <0%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8830ce2...4fb16c9. Read the comment docs.

sibiryakov commented 6 years ago

Hi @icapurro thank you for the contribution! Since this is just an incompatibility of API's then it's quite unlikely this will happen next in exactly the same place. So I propose to remove the test case, otherwise it looks fine.

jpbalarini commented 6 years ago

👍

icapurro commented 6 years ago

@sibiryakov thanks for the review. I removed the unit test; my thought was to add some unit test structure so they can be implemented more in-depth in the future.

sibiryakov commented 6 years ago

thank you!