luqmaan / PriceChecker.py

A web server + scraper that notifies you when the price of products change.
3 stars 1 forks source link

Determining the selector #13

Open luqmaan opened 11 years ago

luqmaan commented 11 years ago

So the approaches for getting the selector:

We've got a decision to make.

jmfield2 commented 11 years ago

So, the second or last option is probably the best so far -- the first will be limited by cross-domain javascript restrictions as far as I can tell.. And since phantom supports clicking on an area, we should be able to find the right or nearby elements without much trouble...the only problem with that might be whether we have to load phantom each time they "click" and how well it or we can cache the relevant stuff..

Maybe we should consider "hard-coding" some of the sites as well because then we can make it extract multiple blocks by default similar to how iftt.com works ... Maybe the chrome extension can be used to make new "recipes" ... And that brings us to the "that" part ... With multiple blocks being returned from each 'recipe' we could use it to render the selected action {{price}} or whatever... and somehow it would be neat to allow users to customize and create those too. I like the idea of the extension because it could let us offload some of the processing to clients say if they only wanted to save something locally...

On Tue, Apr 30, 2013 at 1:16 PM, Luqmaan Dawoodjee <notifications@github.com

wrote:

So the approaches for getting the selector:

  • Anything: Get the html for the page, insert it in a div and bind the selector js to its elements
  • Price: Get the text of the page, use a regex to find all the prices on it. Present those prices back to the user and let them click on the one that is the real price. Search the html for the selector to the price.
  • Price: Handpicked.
  • Anything: Open the page in phantom. Take a screenshot and present it to the user. Let the user click in the area where they see the price on the image. Inject our selector js into the page. Emulate a click in that area and get the selector for it.

— Reply to this email directly or view it on GitHubhttps://github.com/createch/PriceChecker.py/issues/13 .