search

fergiemcdowall / norch-fetch

Fetch pure HTML from a webserver and save it to disk
MIT License
8 stars 2 forks source link

Cannot get norch-fetch to fetch/index anything #16

Open stephenwil opened 9 years ago

stephenwil commented 9 years ago

Running the command returns immediately with undefined; Debugging shows it seems to fail around the initial get. Tried on both windows 10 & OSX platforms.

eklem commented 9 years ago

I know that it doesn't tackle redirects, could it be that?

eklem commented 9 years ago

Here's the issue with not following redirects: https://github.com/fergiemcdowall/norch-fetch/issues/12

Also, norch-fetch is maybe the most immature part of Norch. I would maybe consider using Kimono Labs for my crawling/screenscraping needs. At least for now. https://www.kimonolabs.com/

fergiemcdowall commented 9 years ago

It does seem to be somehow broken. We need to set up tests.