search

elixir-crawly / crawly

Crawly, a high-level web crawling & scraping framework for Elixir.
https://hexdocs.pm/crawly
Apache License 2.0
982 stars 115 forks source link

[Draft] Add ui module retrival from config #162

Closed FrancescoZ closed 3 years ago

FrancescoZ commented 3 years ago

Add a new config to notify a different module when an Item is created.

The pipeline still using CrawlyUI as a default value if the configuration is not set

This resolve #152

FrancescoZ commented 3 years ago

@oltarasenko Not sure how I should test this. I cannot us meck module to test the call, any suggestion?

Ziinc commented 3 years ago

@FrancescoZ is :rpc one of the erlang modules that cannot be mocked?

One way to test this (that i can think of) is to send it to itself, then have a mocked module passed as the ui_module where you can then verify that store_item was called.

oltarasenko commented 3 years ago

Looks good to me. If it's possible to add a test that tries to send an rpc call to itself, it could be beneficial. (Otherwise, I would not try to mock RPC).

oltarasenko commented 3 years ago

@FrancescoZ I still have a bit of development time in my timeslot. Should I take over this task? (In case if you're stuck with other things)

oltarasenko commented 3 years ago

I have decided to abandon the CrawlyUI project for now. I don't have any spare time to maintain both projects, so I think this one is not a valid thing anyway.