search

nietaki / crawlie

A simple Elixir library for writing decently-performing crawlers with minimum effort.
MIT License
89 stars 11 forks source link

limiting urls to a domain #5

Closed nietaki closed 7 years ago

nietaki commented 7 years ago

It's now easy to do it in the crawler logic, add an example in the crawlie_example project instead.

nietaki commented 7 years ago

Added an example in https://github.com/nietaki/crawlie_example/pull/2 , will simplify it a bit after changing the Response struct fields.