Starting run_crawl from page=1 with start_page(x) = x*10+1 results in engines starting from 11th result. Also added limit to arxiv and fixed token regex in djs
Changes
page = 1 and start_page(x) = (x-1)*10+1 in run_crawl of bing, duckduckgo, millionshort, qwant, baidu, yahoo and exalead.
Limit added in arxiv
Fixes djs regex.
Checklist
[X] I have read the Contribution & Best practices Guideline.
[X] My branch is up-to-date with the Upstream master branch.
[ ] The acceptance, integration, unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)
Resolves
Starting run_crawl from page=1 with
start_page(x) = x*10+1
results in engines starting from 11th result. Also added limit to arxiv and fixed token regex in djsChanges
page = 1
andstart_page(x) = (x-1)*10+1
in run_crawl of bing, duckduckgo, millionshort, qwant, baidu, yahoo and exalead.Checklist
master
branch.