1.For all the crawlers ,client needs only data not more than 2012.Because,crawling the data before 2012,becoming load high in machine as well as in elasticsearch.
2.So that ,we have changed the crawlers by keeping the condition as publish_time not more than 2012.
1.For all the crawlers ,client needs only data not more than 2012.Because,crawling the data before 2012,becoming load high in machine as well as in elasticsearch. 2.So that ,we have changed the crawlers by keeping the condition as publish_time not more than 2012.