search

gawel / pyquery

A jquery-like library for python
http://pyquery.rtfd.org/
Other
2.3k stars 182 forks source link

Why I must wait some seconds, the pyquery can return the right result. #71

Closed viquu closed 10 years ago

viquu commented 10 years ago

I want track a Lego at amazon.co.jp.

My code is : 

amzJp42009Url = 'http://www.amazon.co.jp/gp/product/B00B0IDCJM/'
userAgent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36'

headers = {'User-Agent' : userAgent}

r = requests.get(amzJp42009Url, headers=headers)

html = r.text
d = pq(html)

print d.html()   //something else like: time.sleep(1)

price = d("#actualPriceValue").html()

print price

The this code can get the price.But if I remove the line : 

print d.html()  //something else like: time.sleep(1)

Then the price is None.

gawel commented 10 years ago

This code always show me:

<b class="priceLarge">￥ 18,507  &#13;
&#13;
</b>

Even with time.sleep(1) or nothing or whatever

gawel commented 10 years ago

Oh no, you're right, weird things happens.. I don't have a clue

gawel commented 10 years ago

Ok, I got one:

$ python p.py
[<html>]  # this is repr(d)
<b class="priceLarge">￥ 18,507  &#13;
&#13;
</b>
$ python p.py
[<html.a-no-js>]   # this is repr(d)
None

The html output is not always the same. I guess the behavior of the url is sometime different.

But, I thinks it's more a requests/website issue than a pyquery issue

viquu commented 10 years ago

oh, thanks. I will check the website return. 2014年5月31日 上午4:30于 "Gael Pasgrimaud" notifications@github.com写道：

Ok, I got one:

$ python p.py [] # this is repr(d) ￥ 18,507 $ python p.py [] # this is repr(d) None

The html output is not always the same. I guess the behavior of the url is sometime different.

But, I thinks it's more a requests/website issue than a pyquery issue

— Reply to this email directly or view it on GitHub https://github.com/gawel/pyquery/issues/71#issuecomment-44696087.