vezinaca / Banq_Achat

Divers scripts qui interagissent avec la BANQ
0 stars 0 forks source link

Search on banq site not working because of %20 #35

Open vezinaca opened 4 years ago

vezinaca commented 4 years ago

The search text from user input is for example: 'Frank Zappa'.

Link before sending to request.get() is: http://www.banq.qc.ca/techno/recherche/rms.html?q=Frank Zappa

but when I have webbrowser.open(link), search field shows as Frank%20Zappa and banq search engine doesn't find anything (0 results).

gregsadetsky commented 4 years ago

spaces need to be escaped in URLs. this is also called 'url encoding'

you can see a relevant question here

vezinaca commented 4 years ago
pre_url_search = 'http://www.banq.qc.ca/techno/recherche/rms.html'
param_search = {'q': search_text}

def get_response(parametres):
    res = requests.get(pre_url_search, params=parametres)
    res.raise_for_status()
    return res
response = get_response(param_search)

keeping this open as still can't figure out way for webbrowser.open(url)

vezinaca commented 4 years ago

The previous version of this code worked fine last time I used it (>a week ago?) Could the banq have changed something on site?

vezinaca commented 4 years ago

of course they could have changed something on site, but was this particular 'issue' a security one?

gregsadetsky commented 4 years ago

what result do you get from the banq? if you dump the html that you're getting, what's in it?

vezinaca commented 4 years ago

banq-recherche2

vezinaca commented 4 years ago

can't really dump html...this is the webbrowser.open() function: I get this when I try both calls (even by artificially adding %20): webbrowser.open('http://www.banq.qc.ca/techno/recherche/rms.html?q=Frank%20Zappa') webbrowser.open('http://www.banq.qc.ca/techno/recherche/rms.html?q=Frank Zappa')

gregsadetsky commented 4 years ago

webbrowser.open and requests.get are two completely different beasts

it's possible that webbrowser.open passes the url and does urlencoding itself -- you would pass the url with a space. requests.get will not urlencode automatically...

those are 2 separate issues.

dump the html from requests.get and check if the results are there or not. do not use webbrowser.open to debug requests, that's just adding an extra layer of confusion