search

taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.39k stars 581 forks source link

How to get tweet geolocation #109

Open ghost opened 6 years ago

ghost commented 6 years ago

Hi, How to get the tweet geolocations in the scrapped data?

lapp0 commented 6 years ago

https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators.html

When you want to know what Tweets are coming from a specific location, with a specific language:

You want: all recent Tweets in Portuguese, near Maracanã soccer stadium in Rio de Janeiro

Your search URL is: https://api.twitter.com/1.1/search/tweets.json?q=geocode=-22.912214,-43.230182,1km&lang=pt&result_type=recent

Geolocalization: the search operator “near” isn’t available in the API, but there is a more precise way to restrict your query by a given location using the geocode parameter specified with the template “latitude,longitude,radius”, for example, “37.781157,-122.398720,1mi”. When conducting geo searches, the search API will first attempt to find Tweets which have lat/long within the queried geocode, and in case of not having success, it will attempt to find Tweets created by users whose profile location can be reverse geocoded into a lat/long within the queried geocode, meaning that is possible to receive Tweets which do not include lat/long information.

rafaeloliveiras commented 5 years ago

Is there any chance to get a Tweet geolocation information? I'm doing my best but the field comes blank. Can you help me on this?

taspinar commented 5 years ago

@rafaeloliveiras You can get location in profile if filled in. Have a look at #46 and #63

rafaeloliveiras commented 5 years ago

@rafaeloliveiras You can get location in profile if filled in. Have a look at #46 and #63

Thank you @taspinar ! I'm looking at it and i didnt find out how get geo user data in that code. Can i easily get unique users from a csv for example in a specific column[usernames] and fill the locations in a new single table [geo] ?

taspinar commented 5 years ago

It is not possible to get user geo data, but it is possible to scrape the location from the profile and turn that into geo data. You can do that with the script in #46.

What you're asking now seems more like a general programming question... The quickest way would be to load the .csv file into a pandas dataframe and get the unique values from the specified column, like:

df = pd.read_csv('tweets.csv', index_col=0)
unique_users = df['usernames'].unique()