Setting up a parametric spider (dynamic base_url and start_urls)

elixir-crawly / crawly

Crawly, a high-level web crawling & scraping framework for Elixir.

https://hexdocs.pm/crawly

Apache License 2.0

976 stars 115 forks source link

Closed matteoredaelli closed 1 year ago

matteoredaelli commented 2 years ago

Hello

How could I develop a parametric spider and run it like

Crawly.Engine.start_spider(MySpider, base_url, start_urls)

Thanks in advance Matteo

Ziinc commented 1 year ago

Please use spider init/1 arguments. Reading the documentation is also advised.