alephdata / memorious

Lightweight web scraping toolkit for documents and structured data.
https://docs.alephdata.org/developers/memorious
MIT License
311 stars 59 forks source link

Depth first execution in sync mode #166

Closed sunu closed 3 years ago

sunu commented 3 years ago

fixes #165

pudo commented 3 years ago

Upon consideration, I'd like to make the argument against this change:

So, arguably, we're introducing a whole new order of magnitude of complexity to enable something that's pretty smelly behaviour to begin with (generating a million tasks in memory).