Exception: Invalid field name: Date! friends_followers__calculate_avg_influence_of_followers.py

[constzi]

I get an Exception: Invalid field name: Date!

when I run: friends_followers__calculate_avg_influence_of_followers.py

It seems it does not like the fields at the line of code:

fields = ['Date', 'Count']

Tried to remove date, but then there error was:

I get an Exception: Invalid field name: Count!

Any input will greatly help.

[ptwobrussell]

Can you post a full stack trace? Thanks.

[constzi]

Hi Matthew,

Thank you for your reply. Love your book btw.

I added the trackback code and I saw in the error that I forgot to easy_install the psyco library. So I tried to easy_install psyco on both my win7 64bit and ubuntu 12.04 32 bit and got the following error:

Searching for psyco Reading http://pypi.python.org/simple/psyco/ Reading http://psyco.sourceforge.net/ Best match: psyco snapshot Downloading http://wyvern.cs.uni-duesseldorf.de/psyco/psyco-snapshot.tar.gz error: Download error for http://wyvern.cs.uni-duesseldorf.de/psyco/psyco-snapshot.tar.gz: [Errno 113] No route to host

The path does no longer exist: http://wyvern.cs.uni-duesseldorf.de/psyco/psyco-snapshot.tar.gz, so manually installed the .exe for python2.5 on my windows 7 64bit (on which I have python2.7 installed) - I got the file from http://psyco.sourceforge.net/psycoguide/sources.html at the following link http://sourceforge.net/project/showfiles.php?group_id=41036 .

Now I get the following error on my PC -I guess that psyco is not compatible with version 2.7: C:\Users\ccc\Dropbox\Documents\const\class\python_code>friends_followerscalculate_avg_influence_of_followers.py constantinz Traceback (most recent call last):   File "C:\Users\ccc\Dropbox\Documents\const\class\python_code\friends_followerscalculate_avg_influence_of_followers.py", line 68, in     import psyco   File "C:\Python27\lib\site-packages\psycoinit.py", line 46, in     raise ImportError, str(e) + extramsg ImportError: DLL load failed: The specified module could not be found. (check that the compiled extension 'C:\Python27\lib\site-packages\psyco_psyco.pyd' is for the correct Python version; this is Python 2.7.2)Traceback (most recent call last):   File "C:\Users\ccc\Dropbox\Documents\const\class\python_code\friends_followerscalculate_avg_influence_of_followers.py", line 68, in     import psyco   File "C:\Python27\lib\site-packages\psycoinit__.py", line 46, in     raise ImportError, str(e) + extramsg ImportError: DLL load failed: The specified module could not be found. (check that the compiled extension 'C:\Python27\lib\site-packages\psyco_psyco.pyd' is for the correct Python version; this is Python 2.7.2)

I did a search on this issue, and found that it psyco is no longer supported on the new version of python, if i am correct, and an alternative is pypy: http://pypy.org/ (here is the info I found http://stackoverflow.com/questions/8599162/i-use-python-2-7-windows-7-64-bit-alternatives-to-psyco )

If you can help me make this code work on python 2.7 on either my ubuntu or pc that would be great. Of course a solution for both platforms would be fantastic :)

Thank you for your time,

Const

---

From: Matthew A. Russell reply@reply.github.com To: constzi constantinz@yahoo.com Sent: Monday, July 2, 2012 7:43 AM Subject: Re: [Mining-the-Social-Web] Exception: Invalid field name: Date! friends_followers__calculate_avg_influence_of_followers.py (#30)

Can you post a full stack trace? Thanks.

---

Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/30#issuecomment-6709940

[constzi]

Sorry for the follow up, I just realized that your code uses psyco when available. 

I adjusted the code with the following addition in bold, to trace this - not sure if this is correct way, sorry I am a Python newbie:

    fields = ['Date', 'Count']     try:     pt = PrettyTable(fields=fields)     except:                 import traceback         traceback.print_exc()

So by bypassing psyco, I get the same error below: 

C:\Users\ccc\Dropbox\Documents\const\class\python_code>friends_followerscalculate_avg_influence_of_followers.py constantinz The top 10 followers from the sample: Traceback (most recent call last):   File "C:\Users\ccc\Dropbox\Documents\const\class\python_code\friends_followerscalculate_avg_influence_of_followers.py", line 52, in calculate pt = PrettyTable(fields=fields)   File "build\bdist.win32\egg\prettytable.py", line 125, in init self._validate_option(option, kwargs[option])   File "build\bdist.win32\egg\prettytable.py", line 210, in _validate_option self._validate_all_field_names(option, val)   File "build\bdist.win32\egg\prettytable.py", line 285, in _validate_all_field_names self._validate_field_name(name, x)   File "build\bdist.win32\egg\prettytable.py", line 280, in _validate_field_name raise Exception("Invalid field name: %s!" % val) Exception: Invalid field name: Date!

Traceback (most recent call last):   File "C:\Users\ccc\Dropbox\Documents\const\class\python_code\friends_followerscalculate_avg_influence_of_followers.py", line 83, in  calculate()   File "C:\Users\ccc\Dropbox\Documents\const\class\python_code\friends_followerscalculate_avg_influence_of_followers.py", line 52, in calculate  pt = PrettyTable(fields=fields)   File "build\bdist.win32\egg\prettytable.py", line 125, in init   File "build\bdist.win32\egg\prettytable.py", line 210, in _validate_option   File "build\bdist.win32\egg\prettytable.py", line 285, in _validate_all_field_names   File "build\bdist.win32\egg\prettytable.py", line 280, in _validate_field_name Exception: Invalid field name: Date!

---

From: Matthew A. Russell reply@reply.github.com To: constzi constantinz@yahoo.com Sent: Monday, July 2, 2012 7:43 AM Subject: Re: [Mining-the-Social-Web] Exception: Invalid field name: Date! friends_followers__calculate_avg_influence_of_followers.py (#30)

Can you post a full stack trace? Thanks.

---

Reply to this email directly or view it on GitHub: https://github.com/ptwobrussell/Mining-the-Social-Web/issues/30#issuecomment-6709940

[constzi]

I think the problem is with the PrettyTable code, since when I comment out the code of pretty table and just do a print the code works. Let me know what you find out. Below is my code that works, without PrettyTable:  ...  

        #pt = PrettyTable(fields=fields)
    #raise# psyco can only compile functions, so wrap code in a function
    #[pt.set_field_align(f, 'l') for f in fields]

    for (user, freq) in reversed([(user['screen_name'], k) for k in keys[-10:]
                    for user in freqs[k]]):
    #pt.add_row([user, pp(freq)])
    print user + " - " + pp(freq)

    #pt.printt()

    all_freqs = [k for k in keys for user in freqs[k]]
    avg = reduce(lambda x, y: x + y, all_freqs) / len(all_freqs)

    print "\nThe average number of followers for %s's followers: %s" \
    % (SCREEN_NAME, pp(avg))

...

[ptwobrussell]

Based on what I am seeing from your output, my best guess is that something may be wrong with how psyco is trying to optimize code within the PrettyTable package. If you were to remove the try/catch block entirely and execute calculate() directly without trying to do anything at all with psyco (namely, bind it to the calculate() function), the code would also work. Would be interested to know if this works out for you. (I seriously doubt that PrettyTable is doing anything to cause an error, and I didn't see anything in your stack traces that suggested that it was at fault. At any rate, I'm gal that this is working for you.)

[constzi]

Thanks for the quick reply. Fyi, I did remove the psyco completely and I got the same error, until I commented out the prettytable code, and added the print statements, like I showed you on the code above.

What is weird is that prettytable works when I compile a simple app as follow, but not with your code :(

# -*- coding: utf-8 -*-
from prettytable import PrettyTable
x = PrettyTable(["City name", "Area", "Population", "Annual Rainfall"])
x.add_row(["Adelaide",1295, 1158259, 600.5])
x.add_row(["Sydney", 2058, 4336374, 1214.8])
x.add_row(["Melbourne", 1566, 3806092, 646.9])
print x

Does you code compile on your machine? Reason I ask is that the code below is not liked on either Win7 or Ubuntu:

fields = ['Date', 'Count']
pt = PrettyTable(fields=fields)

Could it be something with the syntax because of older python version?

[ptwobrussell]

The code that you provided does run just fine on my machine with prettytable version 0.5 (as shown by printing out

pretty table.__file__

) with Python 2.6, but it could be entirely possible that prettytable has changed its constructor since then if you're running a newer version, which seems to be 0.6.1 per PyPi http://pypi.python.org/pypi/PrettyTable. The change should be pretty easy to incorporate if the constructor is different in a newer version.

This is another case in point for me needing to get a VM setup with the right version of the code checked out on it. As these OSS libs change, the code is increasingly breaking. Lots of these scripts use pretty table, so I might have to do a larger review of the code and see what else is broken at this point. I appreciate you bringing this to light.

[StewartNoyce]

Matthew,

Great work on the book.

I ran into the same issue in Example 6.2 (linkedin_analyze_titles.py). With some investigation, it appears that PrettyTable 0.6.1. broke compatibility with 0.5. This works in Python 2.6:

pt = PrettyTable(['Token', 'Freq']) pt._set_align('l') print pt

Check it out.

Stewart

[ptwobrussell]

I really appreciate the confirmation/follow up on this. I'll make a note to go through and patch the scripts that use the prettytable package very soon. In the meanwhile, patches would be welcome (for anybody who wants to take a crack at these relatively simple patches during the course of their own work.) I'll follow up here one last time once it's all patched.

[robbfitzsimmons]

Hi Matthew and others,

Here's my patch, which is helpful for those who might not know how to integrate Stewart's lines. Can just be subbed line-for-line from 49-58 and works with PrettyTable 0.6.1/Python 2.7.3.

fields = ['Date', 'Count'] pt = PrettyTable(field_names=fields) for f in fields: pt.align = "l"

for (user, freq) in reversed([(user['screen_name'], k) for k in keys[-10:] for user in freqs[k]]): pt.add_row([user, pp(freq)])

print pt.get_string()

Thanks and great work on this book. Robb

[sidmenon]

Hi Matthew,

I got the same error "Invalid Field Name: Company" error for Example 6-1 (linkedin_analyze_companies.py) and I'm not able to resolve this error despite trying the different approaches mentioned here.

Can I paste my output here - since it's the same type of error, only for a different script - to get your help on this? This is my first time posting on Github so, please do guide me if that's not the right way of opening an issue here!

Thanks,

Sid

[ptwobrussell]

Sure, feel free to post in your script, but more importantly, can you also include your version of PrettyTable as well? AFAICT, that's going to be the real issue and any fix is going to come down to being able to tweak the code to work as robbfitzsimmons describes. Glad to help though if you need a hand, so let me know if you haven't gotten this to work yet.

[sidmenon]

Matthew,

Thank you for getting back to me! I appreciate it a lot.

Quick notes:

1. I have PrettyTable 0.6.1.
2. This is regarding Example 6-1 (linkedin_analyze_companies.py)
3. After changing directories to get to the directory that contains the CSV file, I ran line #15.

In [15]: run linkedin_analyze_companies.py linkedin_connections_export_microsoft_outlook.csv

Exception Traceback (most recent call last) /Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/IPython/utils/py3compat.py in execfile(fname, where) 173 else: 174 filename = fname --> 175 builtin.execfile(filename, where)

/Users/Menon/Documents/Python Stuff/PyScripts/Textbook/linkedin_analyze_companies.py in () 21 companies[i] = companies[i].replace(*transform) 22 ---> 23 pt = PrettyTable(fields=['Company', 'Freq']) 24 pt.set_field_align('Company', 'l') 25 fdist = nltk.FreqDist(companies)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/prettytable.pyc in init(self, field_names, encoding, **kwargs) 123 for option in self._options: 124 if option in kwargs: --> 125 self._validate_option(option, kwargs[option]) 126 else: 127 kwargs[option] = None

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/prettytable.pyc in _validate_option(self, option, val) 208 self._validate_hrules(option, val) 209 elif option in ("fields"): --> 210 self._validate_all_field_names(option, val) 211 elif option in ("header", "border", "reversesort"): 212 self._validate_true_or_false(option, val)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/prettytable.pyc in _validate_all_field_names(self, name, val) 283 try: 284 for x in val: --> 285 self._validate_field_name(name, x) 286 except AssertionError: 287 raise Exception("fields must be a sequence of field names!")

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/prettytable.pyc in _validate_field_name(self, name, val) 278 assert val in self._field_names 279 except AssertionError: --> 280 raise Exception("Invalid field name: %s!" % val) 281 282 def _validate_all_field_names(self, name, val):

Exception: Invalid field name: Company!

------------------------------------------------------------------------- END --------------------------------------------------------------------------------

Does this help identify the issue?

[asmello]

The following code for mailboxes__count_json_mbox_by_date_time.py works fine for me, using prettytable 0.6.1:

fields = ['Date', 'Count']
pt = PrettyTable(field_names=fields)
pt.align = 'l'

for row in db.view('index/doc_count_by_date_time', group_level=3):
    pt.add_row(['-'.join([str(i) for i in row.key]), row.value])

print pt.get_string()

[ptwobrussell]

I believe the PrettyTable issues have been fixed with recent changesets, so I'm going to close this.

[femibyte]

Can you tell me what versions of modules I need to make this work? I upgraded to prettytable 0.7.2 and I still get this error: pt = PrettyTable(fields=fields) File "/Library/Python/2.7/site-packages/prettytable.py", line 132, in init self._validate_option(option, kwargs[option]) File "/Library/Python/2.7/site-packages/prettytable.py", line 270, in _validate_option self._validate_all_field_names(option, val) File "/Library/Python/2.7/site-packages/prettytable.py", line 383, in _validate_all_field_names self._validate_field_name(name, x) File "/Library/Python/2.7/site-packages/prettytable.py", line 378, in _validate_field_name raise Exception("Invalid field name: %s!" % val) Exception: Invalid field name: Date!

[ptwobrussell]

Hmm. That version should work, though I think the version I used is 0.7.1. Can you tell me the name of the script where you are seeing this error. It could be possible that I missed one of the scripts when I was going through and upgrading the code to work with the new prettytable syntax

[megansquire]

I am using prettytable 0.7.2 and I get the following when running the Example 3-14 threading code (https://github.com/ptwobrussell/Mining-the-Social-Web/blob/master/python_code/mailboxes__threading.py)

Traceback (most recent call last): File "ex3-14.py", line 59, in pt = PrettyTable(fields=fields) File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/prettytable-0.7.2-py2.7.egg/prettytable.py", line 132, in init self._validate_option(option, kwargs[option]) File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/prettytable-0.7.2-py2.7.egg/prettytable.py", line 270, in _validate_option self._validate_all_field_names(option, val) File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/prettytable-0.7.2-py2.7.egg/prettytable.py", line 383, in _validate_all_field_names self._validate_field_name(name, x) File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/prettytable-0.7.2-py2.7.egg/prettytable.py", line 378, in _validate_field_name raise Exception("Invalid field name: %s!" % val) Exception: Invalid field name: Thread Id!

Based on previous postings in this thread, I changed the code as follows:

fields = ['Thread Id', 'Thread Length']

pt = PrettyTable(fields=fields)

[pt.set_field_align(f, 'l') for f in fields]

pt = PrettyTable(field_names = fields) pt.align='l'

for stat in stats: pt.add_row(stat)

pt.printt()

print pt.get_string()