search

crawlserv / crawlservpp

crawlserv++: Application for crawling and analyzing textual content of websites.
Other
5 stars 0 forks source link

extractor: option to get multiple pages from each page #112

Closed crawlserv closed 4 years ago

crawlserv commented 4 years ago

needed for recursive paging

e.g. get first page with all comment roots -> request one page with replies for every comment root --> request one page with replies for every reply

crawlserv commented 4 years ago

implemented as recursive paging