num_results_per_page is not working as expected

ecoron / SerpScrap

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

MIT License

257 stars 61 forks source link

Hi,

First of all thank you for making such a great library.

I am trying to play around with it using python 3.

Below is my sample code but i am not able to limit the number of searches.

import pprint
import serpscrap

keywords = ['Virendra Sahewag']

config = serpscrap.Config()
config.set('scrape_urls', False)
config.set('num_pages_for_keyword', 2)
config.set('num_results_per_page', 3)

scrap = serpscrap.SerpScrap()
scrap.init(config=config.get(), keywords=keywords)
results = scrap.run()

for result in results:
    pprint.pprint(result)

I tried to change num_pages_for_keyword and num_results_per_page but in most of case, it is giving me 30 results always.

If anyone can help me about it.

Thanks! Jayesh

ecoron / SerpScrap

num_results_per_page is not working as expected #55