Does it make sense to have support for headless crawling built-in to the framework? A lot of the websites these days are Single Page apps and crawling that using the current framework won't work.
We could try to do this using phantonjs or capybara-webkit. I've been able to do a headless crawl using capybara-website and poltergeist before.
Does it make sense to have support for headless crawling built-in to the framework? A lot of the websites these days are Single Page apps and crawling that using the current framework won't work.
We could try to do this using phantonjs or capybara-webkit. I've been able to do a headless crawl using capybara-website and poltergeist before.
What are your thoughts on this?