kltm / tanuki

A novelty seeking crawler for HTML-based web applications.
BSD 3-Clause "New" or "Revised" License
1 stars 2 forks source link

Add agents that they can spider dynamic content #1

Open kltm opened 9 years ago

kltm commented 9 years ago

Currently, tanuki can only interact with static content. An agent should be added that can access dynamic JS content (maybe waiting a few seconds before processing or waiting for connection quiescence).

Since there don't seem to be many native methods for this, possibilities include:

kltm commented 9 years ago

Actually, there may be a way through this with Electron as well? https://news.ycombinator.com/item?id=9877314