Add webscraper

[juancarlospaco]

• webscraper can help implement a basic web scraper with standard library.
• Documentation, examples, runnableExamples with doAssert, since, etc, etc.
• Works at compile time. Simple API, easy to extend with nimble or stdlib procs.

[juancarlospaco]

Ping @Araq :)

[Araq]

The scrap iterators are exported and have no documentation comments. In fact, you made the documentation worse.

[disruptek]

ITYM "scrape", not "scrap"?

[juancarlospaco]

@disruptek I dont care too much about the name, give it a name I will rename it. :)

[disruptek]

Done and done.

[Araq]

For the exported iterators body should be of type XmlNode. If you then need a more convenient parseHtml proc, add one.

[juancarlospaco]

@Araq Done, added more docs, and changed as you suggested. :)

[Araq]

Sorry for not mentioning this earlier but this module doesn't look like it was used in production and it seems to me it tries to solve hypothetical problems. I wouldn't use it for anything and please note that I actually did write web crawlers in the past.

[juancarlospaco]

@Araq What does that mean?. You want more runnableExamples ?. Feel free to send Pull Request for another version, is not the only way of making a scraper, it can be more "your way", more verbose if you like that way, more OOP-ish, etc etc.

Ive got some people to try Nim by just showing this:

(That version has 1 extra line newHttpClient().getContent(url) but is same code) People liked it, etc, so I though I try, after all thats the idea of Fusion.

Or maybe we need a Fusion for Fusion, something like "Contrib". :thinking:

[ghost]

@juancarlospaco "Fusion for Fusion" already exists - publishing packages in Nimble.

[juancarlospaco]

@Yardanico You can say the same for Fusion then. :shrug:

[Araq]

Sorry, this is not for "fusion", this should be a Nimble pacakge. I know it's confusing and we're still figuring out why we have fusion ( :D ) but the idea is that fusion contains stuff we are happy to maintain for a decade to come.