search

gajus / headless-crawler

A crawler implemented using a headless browser (Chrome).
Other
14 stars 0 forks source link

Add a way to prioritise URLs #5

Open gajus opened 5 years ago

gajus commented 5 years ago

i.e. Support both depth-first search and breadth-first search algorithm

gajus commented 5 years ago

This can be done https://github.com/gajus/headless-crawler#headless-crawler-configuration-default-headlesscrawlerconfiguration-sortqueuedlinks.

Add examples prior to closing the issue.

gajus commented 5 years ago

This feature is no longer compatible with the queue mechanism. Needs to be reworked.