serp-spider / search-engine-google

:spider: Google client for SERPS
https://serp-spider.github.io
Other
167 stars 61 forks source link

Add caching feature #49

Closed wntechs closed 7 years ago

wntechs commented 7 years ago

Most of the time a search is conducted using the same keyword and search (google) domain. In this situation there should be cache available so that the results can be obtained instantly without requesting google in real time. The cache data can be deleted after 24 hours timeframe and build new cache after that. This feature will make the spider more efficient and there will less chances of captcha and ban from the search engine. Local file system can be used for this purpose.

gsouf commented 7 years ago

Hi @ramkumarkuril

This kind of feature would produce unexpected results and will force to define a logic inside SERPS that wont suite everyone needs.

This kind of feature is easy to implement and you should be able to place it in front of SERPS with your own solution.