Closed vlaco closed 8 years ago
Is your query including the URL encoding exactly as shown with %2C%20, etc.? If so, I think you should try it without the url encoding. e.g. "migrants OR migrations OR refugees"...
On Thu, Sep 17, 2015 at 7:46 PM, Vanja Vlaco notifications@github.com wrote:
Hi,
I am using the code from the book to use twitter_search and I am trying to use slightly more complex query for my project. The code is running when I use a simple query such as q = 'Migrants' But whenever I try to use q = "migrants%2C%20OR%20migrations%2C%20OR%20refugees%20&geocode=%2244.802416%2C20.465601%2C80mi%22" I get this error print json.dumps(results[0], indent=1) IndexError: list index out of range which I am guessing it means that it is not fining any tweets under these conditions but when I use the same query in Exploring the Twitter API console it is working.
Any ideas what could be wrong please?
— Reply to this email directly or view it on GitHub https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/issues/281 .
You were right, it works without the url encoding, but geocode was my major problem. Luckily I managed to solve it so I will share the code here in case someone else has the same issue. This code is for a case when you want to search for tweets containing specific words and are posted from a specific country.
import oauth_login
import json
def twitter_search(twitter_api, f, p, max_results=200, **kw):
result = twitter_api.geo.search(query = p, granularity="country")
place_id = result['result']['places'][0]['id']
search_results = twitter_api.search.tweets(q="%s place:%s" % (f, place_id), count=100, **kw)
statuses = search_results['statuses']
max_results = min(1000, max_results)
for _ in range(10): # 10*100 = 1000
try:
next_results = search_results['search_metadata']['next_results']
except KeyError, e: # No more results when next_results doesn't exist
break
kwargs = dict([ kv.split('=')
for kv in next_results[1:].split("&") ])
search_results = twitter_api.search.tweets(**kwargs)
statuses += search_results['statuses']
if len(statuses) > max_results:
break
return statuses
\# Sample usage
twitter_api = oauth_login()
p = "Serbia"
f = 'migrants OR migrations OR refugees'
results = twitter_search(twitter_api, f, p, max_results=10)
\# Show one sample search result by slicing the list...
print json.dumps(results[0], indent=1)
Awesome, thanks for sending this. I'll plan to add it as a new receipt to the Twitter Cookbook chapter whenever I get around to making updates.
On Fri, Sep 18, 2015 at 12:18 PM, Vanja Vlaco notifications@github.com wrote:
You were right, it works without the url encoding, but geocode was my major problem. Luckily I managed to solve it so I will share the code here in case someone else has the same issue. This code is for a case when you want to search for tweets containing specific words and are posted from a specific country.
import oauth_login import json
def twitter_search(twitter_api, f, p, max_results=200, _kw): result = twitter_api.geo.search(query = p, granularity="country") place_id = result['result']['places'][0]['id'] search_results = twitter_api.search.tweets(q="%s place:%s" % (f, place_id), count=100, _kw)
statuses = search_results['statuses'] max_results = min(1000, max_results) for _ in range(10): # 10*100 = 1000 try: next_results = search_results['search_metadata']['next_results'] except KeyError, e: # No more results when next_results doesn't exist break kwargs = dict([ kv.split('=') for kv in next_results[1:].split("&") ]) search_results = twitter_api.search.tweets(**kwargs) statuses += search_results['statuses'] if len(statuses) > max_results: break return statuses
Sample usage
twitter_api = oauth_login() p = "USA"
f = 'migrants OR migrations OR refugees'
results = twitter_search(twitter_api, f, p, max_results=10)
Show one sample search result by slicing the list...
print json.dumps(results[0], indent=1)
— Reply to this email directly or view it on GitHub https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/issues/281#issuecomment-141512101 .
No problem, thank you :)
Hi,
I am using the code from the book to use twitter_search and I am trying to use slightly more complex query for my project. The code is running when I use a simple query such as
q = 'Migrants'
But whenever I try to useq = "migrants%2C%20OR%20migrations%2C%20OR%20refugees%20&geocode=%2244.802416%2C20.465601%2C80mi%22"
I get this errorprint json.dumps(results[0], indent=1) IndexError: list index out of range
which I am guessing it means that it is not fining any tweets under these conditions but when I use the same query in Exploring the Twitter API console it is working.Any ideas what could be wrong please?