elixir-crawly / crawly

Crawly, a high-level web crawling & scraping framework for Elixir.
https://hexdocs.pm/crawly
Apache License 2.0
965 stars 113 forks source link

Make WriteToFile pipeline more robust #216

Closed fstp closed 1 year ago

fstp commented 1 year ago

Currently the pipeline will try to write given items to file but if the items do not implement String.Chars it will crash. By using Kernel.inspect any term can be saved without having to explicitly implement the protocol.

This was discovered in an example where the pipeline tried to write Crawly.Request structs but failed due to the struct not implementing String.Chars.

fstp commented 1 year ago

This seems to be the wrong approach, will close this one and make a separate pull request where I implement the String.Chars protocol for the particular type instead.