ptwobrussell
commented
12 years ago Try wrapping the tweet you have in your text file in square brackets. The code is failing on the list comprehension because it is trying to access a value in a list, but your data structure is a dictionary.
Your d = [] has no effect because you immediately overwrite the value of d. Nor does your statement that overwrites the value of d wrap the tweet in square brackets so that a list comprehension can be possible.
Make sense?
On Apr 18, 2012, at 8:20 PM, securemindorg reply@reply.github.com wrote:
First off I would like to say I think this is an absolutly great book, I picked it up a little while ago and just got around to looking at it today. I'm having trouble with example 1.6 in the book. I have a text file with a single tweet in json that looks like:
{"favorited": false, "in_reply_to_user_id": null, "contributors": null, "retweeted_status": {"favorited": false, "in_reply_to_user_id": null, "contributors": null, "truncated": false, "text": "The NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", "created_at": "Tue Apr 03 13:42:00 +0000 2012", "retweeted": false, "in_reply_to_status_id": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_status_id_str": null, "id_str": "187173104408199168", "in_reply_to_screen_name": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/169719095/pfw_background.gif", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg", "profile_sidebar_fill_color": "F6F6F6", "is_translator": false, "id": 42703684, "profile_tex t_color": "333333", "followers_count": 18662, "protected": false, "location": "Riverwoods, Illinois", "default_profile_image": false, "listed_count": 768, "utc_offset": -21600, "statuses_count": 9397, "description": "The Authority on Pro Football - \r\nBecome a fan on Facebook at www.facebook.com/ProFootballWeekly", "friends_count": 329, "profile_link_color": "038543", "profile_image_url": "http://a0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg", "notifications": null, "show_all_inline_media": true, "geo_enabled": false, "profile_background_color": "545c67", "id_str": "42703684", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/169719095/pfw_background.gif", "screen_name": "ProFootballWkly", "lang": "en", "profile_background_tile": true, "favourites_count": 0, "name": "Pro Football Weekly", "url": "http://www.ProFootballWeekly.com", "created_at": "Tue May 26 19:52:34 +0000 2009", "contributors_enabled": false, "time_zone": "Central Time (US & Canada)", "profile_sidebar_border_color": "EEEEEE", "default_profile": false, "following": null}, "place": null, "retweet_count": 5, "geo": null, "id": 187173104408199168, "source": "<a href=\"http://www.tweetdeck.com\" rel=\"nofollow\">TweetDeck</a>"}, "truncated": false, "text": "RT @ProFootballWkly: The NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", "created_at": "Tue Apr 03 13:51:42 +0000 2012", "retweeted": false, "in_reply_to_status_id": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [{"indices": [3, 19], "screen_name": "ProFootballWkly", "id": 42703684, "name": "Pro Football Weekly", "id_str": "42703684"}], "hashtags": [], "urls": []}, "in_reply_to_status_id_str": null, "id_str": "187175547175059456", "in_reply_to_screen_name": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_ima ges/431674307/vbgiants_545_140211.jpg", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg", "profile_sidebar_fill_color": "211d18", "is_translator": false, "id": 125403702, "profile_text_color": "736d66", "followers_count": 1373, "protected": false, "location": "Queretaro, Mexico", "default_profile_image": false, "listed_count": 15, "utc_offset": -21600, "statuses_count": 15347, "description": "Esposo, Padre, hermano de muchos, 49er y SFGiant de nacimiento,Detroit Piston y Red Wing. colaborador en gradacentral.com de NFL y MLB", "friends_count": 1798, "profile_link_color": "f9a65e", "profile_image_url": "http://a0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": false, "profile_background_color": "000000", "id_str": "125403702", "profile_background_image_url": "http://a0.twimg.com/profile_background_image s/431674307/vbgiants_545_140211.jpg", "screen_name": "asosa49", "lang": "es", "profile_background_tile": false, "favourites_count": 31, "name": "alvaro sosa", "url": "http://www.49ers.com", "created_at": "Mon Mar 22 18:04:40 +0000 2010", "contributors_enabled": false, "time_zone": "Mexico City", "profile_sidebar_border_color": "413d39", "default_profile": false, "following": null}, "place": null, "retweet_count": 5, "geo": null, "id": 187175547175059456, "source": "web"}I'm trying to load the text field values so that I can move on to counting them (example 1.7). I'm trying the following code as shown / interpreted from the book:
import json import sys d = [] d = json.loads(open(sys.argv[1]).read()) print d print json.dumps(d, sort_keys=True, indent=4) tweets = [ r['text'] \ for result in d \ for r in result['results'] ] #print tweetsmy output looks like:
python example1p7.py test.json {u'user': {u'follow_request_sent': None, u'profile_use_background_image': True, u'id': 125403702, u'verified': False, u'profile_image_url_https': u'https://si0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg', u'profile_sidebar_fill_color': u'211d18', u'geo_enabled': False, u'profile_text_color': u'736d66', u'followers_count': 1373, u'profile_sidebar_border_color': u'413d39', u'location': u'Queretaro, Mexico', u'default_profile_image': False, u'listed_count': 15, u'utc_offset': -21600, u'statuses_count': 15347, u'description': u'Esposo, Padre, hermano de muchos, 49er y SFGiant de nacimiento,Detroit Piston y Red Wing. colaborador en gradacentral.com de NFL y MLB', u'friends_count': 1798, u'profile_link_color': u'f9a65e', u'profile_image_url': u'http://a0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg', u'notifications': None, u'show_all_inline_media': False, u'profile_background_image_url_https': u'https://si0.twimg.com/profile_bac kground_images/431674307/vbgiants_545_140211.jpg', u'profile_background_color': u'000000', u'id_str': u'125403702', u'profile_background_image_url': u'http://a0.twimg.com/profile_background_images/431674307/vbgiants_545_140211.jpg', u'name': u'alvaro sosa', u'lang': u'es', u'following': None, u'profile_background_tile': False, u'favourites_count': 31, u'screen_name': u'asosa49', u'url': u'http://www.49ers.com', u'created_at': u'Mon Mar 22 18:04:40 +0000 2010', u'contributors_enabled': False, u'time_zone': u'Mexico City', u'protected': False, u'default_profile': False, u'is_translator': False}, u'favorited': False, u'contributors': None, u'retweeted_status': {u'user': {u'follow_request_sent': None, u'profile_use_background_image': True, u'id': 42703684, u'verified': False, u'profile_image_url_https': u'https://si0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg', u'profile_sidebar_fill_color': u'F6F6F6', u'geo_enabled': False, u'profile_text_color': u'333333', u'followe rs_count': 18662, u'profile_sidebar_border_color': u'EEEEEE', u'location': u'Riverwoods, Illinois', u'default_profile_image': False, u'listed_count': 768, u'utc_offset': -21600, u'statuses_count': 9397, u'description': u'The Authority on Pro Football - \r\nBecome a fan on Facebook at www.facebook.com/ProFootballWeekly', u'friends_count': 329, u'profile_link_color': u'038543', u'profile_image_url': u'http://a0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg', u'notifications': None, u'show_all_inline_media': True, u'profile_background_image_url_https': u'https://si0.twimg.com/profile_background_images/169719095/pfw_background.gif', u'profile_background_color': u'545c67', u'id_str': u'42703684', u'profile_background_image_url': u'http://a0.twimg.com/profile_background_images/169719095/pfw_background.gif', u'name': u'Pro Football Weekly', u'lang': u'en', u'following': None, u'profile_background_tile': True, u'favourites_count': 0, u'screen_name': u'ProFootballWkly', u'url ': u'http://www.ProFootballWeekly.com', u'created_at': u'Tue May 26 19:52:34 +0000 2009', u'contributors_enabled': False, u'time_zone': u'Central Time (US & Canada)', u'protected': False, u'default_profile': False, u'is_translator': False}, u'favorited': False, u'contributors': None, u'truncated': False, u'source': u'<a href="http://www.tweetdeck.com" rel="nofollow">TweetDeck</a>', u'text': u"The NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", u'created_at': u'Tue Apr 03 13:42:00 +0000 2012', u'retweeted': False, u'in_reply_to_status_id_str': None, u'coordinates': None, u'id': 187173104408199168, u'entities': {u'user_mentions': [], u'hashtags': [], u'urls': []}, u'in_reply_to_status_id': None, u'in_reply_to_screen_name': None, u'in_reply_to_user_id': None, u'place': None, u'retweet_count': 5, u'geo': None, u'in_reply_to_user_id_str': None, u'id_str': u'187173104408199168'}, u'truncated': False, u'source': u'web', u'text': u"RT @ProFootballWkly: Th e NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", u'created_at': u'Tue Apr 03 13:51:42 +0000 2012', u'retweeted': False, u'in_reply_to_status_id_str': None, u'coordinates': None, u'id': 187175547175059456, u'entities': {u'user_mentions': [{u'indices': [3, 19], u'id_str': u'42703684', u'screen_name': u'ProFootballWkly', u'name': u'Pro Football Weekly', u'id': 42703684}], u'hashtags': [], u'urls': []}, u'in_reply_to_status_id': None, u'in_reply_to_screen_name': None, u'in_reply_to_user_id': None, u'place': None, u'retweet_count': 5, u'geo': None, u'in_reply_to_user_id_str': None, u'id_str': u'187175547175059456'} { "contributors": null, "coordinates": null, "created_at": "Tue Apr 03 13:51:42 +0000 2012", "entities": { "hashtags": [], "urls": [], "user_mentions": [ { "id": 42703684, "id_str": "42703684", "indices": [ 3, 19 ], "name": "Pro Football Weekly", "screen_name": "ProFootballWkly" } ] }, "favorited": false, "geo": null, "id": 187175547175059456, "id_str": "187175547175059456", "in_reply_to_screen_name": null, "in_reply_to_status_id": null, "in_reply_to_status_id_str": null, "in_reply_to_user_id": null, "in_reply_to_user_id_str": null, "place": null, "retweet_count": 5, "retweeted": false, "retweeted_status": { "contributors": null, "coordinates": null, "created_at": "Tue Apr 03 13:42:00 +0000 2012", "entities": { "hashtags": [], "urls": [], "user_mentions": [] }, "favorited": false, "geo": null, "id": 187173104408199168, "id_str": "187173104408199168", "in_reply_to_screen_name": null, "in_reply_to_status_id": null, "in_reply_to_status_id_str": null, "in_reply_to_user_id": null, "in_reply_to_user_id_str": null, "place": null, "retweet_count": 5, "retweeted": false, "source": "<a href=\"http://www.tweetdeck.com\" rel=\"nofollow\">TweetDeck</a>", "text": "The NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", "truncated": false, "user": { "contributors_enabled": false, "created_at": "Tue May 26 19:52:34 +0000 2009", "default_profile": false, "default_profile_image": false, "description": "The Authority on Pro Football - \r\nBecome a fan on Facebook at www.facebook.com/ProFootballWeekly", "favourites_count": 0, "follow_request_sent": null, "followers_count": 18662, "following": null, "friends_count": 329, "geo_enabled": false, "id": 42703684, "id_str": "42703684", "is_translator": false, "lang": "en", "listed_count": 768, "location": "Riverwoods, Illinois", "name": "Pro Football Weekly", "notifications": null, "profile_background_color": "545c67", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/169719095/pfw_background.gif", "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/169719095/pfw_background.gif", "profile_background_tile": true, "profile_image_url": "http://a0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg", "profile_image_url_https": "https://si0.twimg.com/profile_images/313743829/PFW_FB1_normal.jpg", "profile_link_color": "038543", "profile_sidebar_border_color": "EEEEEE", "profile_sidebar_fill_color": "F6F6F6", "profile_text_color": "333333", "profile_use_background_image": true, "protected": false, "screen_name": "ProFootballWkly", "show_all_inline_media": true, "statuses_count": 9397, "time_zone": "Central Time (US & Canada)", "url": "http://www.ProFootballWeekly.com", "utc_offset": -21600, "verified": false } }, "source": "web", "text": "RT @ProFootballWkly: The NFL's new Nike uniforms will be unveiled today at 11 a.m. ET in Brooklyn.", "truncated": false, "user": { "contributors_enabled": false, "created_at": "Mon Mar 22 18:04:40 +0000 2010", "default_profile": false, "default_profile_image": false, "description": "Esposo, Padre, hermano de muchos, 49er y SFGiant de nacimiento,Detroit Piston y Red Wing. colaborador en gradacentral.com de NFL y MLB", "favourites_count": 31, "follow_request_sent": null, "followers_count": 1373, "following": null, "friends_count": 1798, "geo_enabled": false, "id": 125403702, "id_str": "125403702", "is_translator": false, "lang": "es", "listed_count": 15, "location": "Queretaro, Mexico", "name": "alvaro sosa", "notifications": null, "profile_background_color": "000000", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/431674307/vbgiants_545_140211.jpg", "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/431674307/vbgiants_545_140211.jpg", "profile_background_tile": false, "profile_image_url": "http://a0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg", "profile_image_url_https": "https://si0.twimg.com/profile_images/1901950920/asp---AAq2Ez.jpg_large_normal.jpg", "profile_link_color": "f9a65e", "profile_sidebar_border_color": "413d39", "profile_sidebar_fill_color": "211d18", "profile_text_color": "736d66", "profile_use_background_image": true, "protected": false, "screen_name": "asosa49", "show_all_inline_media": false, "statuses_count": 15347, "time_zone": "Mexico City", "url": "http://www.49ers.com", "utc_offset": -21600, "verified": false } } Traceback (most recent call last): File "example1p7.py", line 11, in <module> for r in result['results'] ] TypeError: string indices must be integersI've tried a number of things but nothing seems to work, I always end up with the "TypeError: string indicies must be integers"
Any help would be much appreciated.
Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/21
First off I would like to say I think this is an absolutly great book, I picked it up a little while ago and just got around to looking at it today. I'm having trouble with example 1.6 in the book. I have a text file with a single tweet in json that looks like:
I'm trying to load the text field values so that I can move on to counting them (example 1.7). I'm trying the following code as shown / interpreted from the book:
my output looks like:
I've tried a number of things but nothing seems to work, I always end up with the "TypeError: string indicies must be integers"
Any help would be much appreciated.