kiorky / spynner

Programmatic web browsing module with AJAX support for Python
http://pypi.python.org/pypi/spynner
859 stars 157 forks source link

browser.click("button[name=btnK]",wait_load=True) #5

Closed goog closed 12 years ago

goog commented 12 years ago

when I browser.load("https://www.google.com/webhp?hl=en&tab=ww") and browser.click("button[name=btnK]",wait_load=True) ,but the click didn't work with message: SpynnerJavascriptError: error on click: jq('button[name=btnK]').simulate('click'); Could you give me a hand , thanks in advance!

kiorky commented 12 years ago

use wk_click, see https://github.com/makinacorpus/spynner/blob/master/src/spynner/browser.py#L694

kiorky commented 12 years ago

basically, jquery.simulate which is under the hood of the click() method isnt supported anymore, the .click method is only there for historical and backward compatiblity purpose.

kiorky commented 12 years ago

Be aware that the selector is not a jquery one but a webkit (so more a strick xpath) one.

goog commented 12 years ago

Hi, kiorky! spynner is a great python app! I'm a beginner to use it and want to make requests to google search , I have tried half a day but failed . my code is below: import spynner browser = spynner.Browser() browser.load("http://www.google.com") browser.create_webview() browser.show() browser.wk_fill("input[name=q]", "hola")

In [9]: browser.url

Out[9]: u'http://www.google.com.hk/'

browser.click("input[name=btnK]")

browser.wk_click("input[name=btnK]",wait_load=True)

when it runs to the last line, the ipython seems to do nothing and no result outputs.

goog commented 12 years ago

Hey, pal! I tried browser.sendKeys('input#lst-ib', [Qt.Key_Enter]) just , yet didnt work ,the command line runs without error and the webview has no changes. Maybe there is something error in my development env (ubuntu12.04) or .....(BWT i click the search btn in the webview with mouse , it didn't go)

kiorky commented 12 years ago

[name=*] is not xpath.

kiorky commented 12 years ago

sendkeys is used to send raw keyboard letters, for the record.

goog commented 12 years ago

sorry,I am not familar with xml .XPath uses path expressions to navigate in XML documents (from w3c) . Is there a xml document or is it generated by some function? Pal, could you help me hack a example or give some tutorials? Any guidance would be greatly appreciated.

kiorky commented 12 years ago

import logging
import os
import re
import tempfile

from lxml.html import document_fromstring
from spynner import browser
from PyQt4 import QtCore

FORMAT = '%(asctime)-15s %(message)s'
logging.basicConfig(format=FORMAT)
logging.getLogger('test scrapper').debug("start")
re_flags = re.M|re.U|re.S|re.X
os.environ['DISPLAY'] = os.environ.get('GMB_DISPLAY', ':1')

def get_tree(h):
    """h can be either a zc.testbrowser.Browser or a string."""
    if isinstance(h, file):
        h = h.read()
    if isinstance(h, browser.Browser):
        h = h.html
    if not isinstance(h, basestring):
        h = h.contents
    return document_fromstring(h)

def getQtBrowser(download_directory=None):
    debug=4
    br = browser.Browser(embed_jquery_simulate=False, want_compat=False, debug_level=debug,)
    br.download_directory = download_directory
    return br

def get_url(url, path):
    if not (path.startswith('http://')
            or path.startswith('https://')):
            path = '%s/%s' % (url, path)
    return path

def main(download_directory=None):
    logger = logging.getLogger('scrapper.test')
    if not download_directory:
        download_directory = tempfile.mkdtemp()
    br = getQtBrowser(download_directory)
    br.create_webview()
    br.webview.show()
    br.webview.setWindowState(QtCore.Qt.WindowMaximized)
    br.load('http://www.google.com')
    def can_continuea(abrowser):
        t = get_tree(abrowser)
        return len(t.xpath("//input[@name='q']")) > 0
    br.wait_for_content(can_continuea, 60, u'Timeout while loading account data')
    br.wk_fill('input[name="q"]', 'kiorky')
    t = get_tree(br)
    name = [a.attrib['name']
            for a in  t.xpath('//input[@type="submit"]')
            if 'google' in a.value.lower()][0]
    # search for the search input control which can change id
    input_sel = "input[name='%s']" % name
    # remodve the search live query ...
    br.native_click('input[name="q"]')
    br.wk_click(input_sel)
    def can_continueb(abrowser):
        t = get_tree(abrowser)
        return len( t.xpath('//*[@id="ires"]')) > 0
    br.wait_for_content(can_continueb, 60, u'Timeout while loading account data')
    assert 'cryptelium.net' in br.html
kiorky commented 12 years ago

For the Os environement, just check that your PyQT package is compatible with the QT libraries installed .

kiorky commented 12 years ago

just commited this sample code in the examples folder:

https://github.com/makinacorpus/spynner/blob/master/examples/anothergoogle.py

goog commented 12 years ago

Great ! Thanks very much!

goog commented 12 years ago

Hey,kiorky! this line :br.wk_fill('input[name="q"]', 'kiorky') , if I want to search an item like "谷歌"(a string 'google' in Chinese in utf8) , how to do it?

goog commented 12 years ago

Hi,kiorky! In browser.py line:957 element.evaluateJavaScript("this.value = '%s'" % value) ,when the value is a utf8(not a english query)string,it become messy code in webview , give a hand please!

hakerjack commented 12 years ago

Just found a workaround for this issue. Use trigger method of jQuery instead of simulate.

goog commented 12 years ago

Hey @hakerjack ! a workaround? how to do it?

hakerjack commented 12 years ago

Hi goog,

In the click method from the source code, try replace the jscode: "%s('%s').simulate('click');" to "%s('%s').trigger('click');

goog commented 12 years ago

Thanks! I tried the new version,filling u'string' is OK

kiorky commented 12 years ago

What did you tried ? spynner2.0 or hackerjack comment ? To be clear, Use spynner 2 and refers to pypi docs on how to click.

goog commented 12 years ago

It's spynner2 and a previous anothergoogle.py example

kiorky commented 12 years ago

And have you a problem with it ?

goog commented 12 years ago

now is ok , thx!

LeoHuang2015 commented 10 years ago

if id is a num, like 1,2, browser.click will error!!!!!!1