Typo in Chapter 2, Example 1 Code

[terminator1]

Hi Matthew,

As you know, I'm new to github and all. But in the Chapter 2 code, Example 1 here http://nbviewer.ipython.org/urls/raw.github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/master/ipynb/Chapter2.ipynb

The line is: fields = 'id,name,friends.fields(likes)'

It should be: fields = 'id,name,friends.field(likes)'

Notice that field is singular in the last instance.

Please consider updating the page as its the first example and I'm sure you don't want to crush your reader's confidence the first time around.

Now I'm not sure if I'm supposed to be running code from that page or what. The .zip file has all iPython notebook code. I don't know what that is and I can't invest the time to tackle that mountain at the moment. But since the code in the chapters seems to be straightforward, I'm just going to roll with that for now.

Also in this line: base_url = 'https://graph.facebook.com/me' Please mention that "me" should be replaced by the userid.

Looking forward to hearing your thoughts.

[jwsy]

Hi @terminator1, I'm new to github as well and thanks for contributing!

Check out the Graph API Explorer at https://developers.facebook.com/tools/explorer to try out those queries and you'll see that if you use 'me' and the s in friends you'll get the likes of your friends as in the screenshot below.

If you're somewhat new to python like I am and don't feel like wrestling with the various dependencies during personal time, you might find a lot of value (and saved time) by using the vagrant image and quickstart guide that @ptwobrussell has kindly made for us :).

In order to start the virtual machine, there are just a few easy steps to follow:

• Download and install the latest copy of VirtualBox for your operating system at https://www.virtualbox.org/
• Download and install Vagrant for your operating system at http://www.vagrantup.com/
  • It is highly recommended that you take a moment to read its excellent "Getting Started" guide as a matter of initial familiarization
• Checkout this code repository to your machine using git or with the download links at the top of the main GitHub page.
• Navigate to the 'vagrant' directory in the checkout
• Run the following command: vagrant up

A few additional details once the virtual machine is running:

• What should happen at this point is that Vagrant will use the Vagrantfile that's provided to download a Virtualbox base image and use the commands in the bootstrap.sh script to customize it by installing dependencies that are needed and checking out the latest copy of this GitHub repository. The first time you vagrant up, it may take a few minutes since a base image and updates for it must be downloaded and installed.
• When all of the dependencies are installed, it will start the IPython Notebook server automatically
• Now, point your web browser to http://localhost:8888 and read the instructions in the Chapter0 (Welcome) IPython Notebook to get started running the code!

At this point, I get my app tokens/keys and shift-enter through the IPython notebook :)

Jason

[ptwobrussell]

Thanks @jwsy for the help with @terminator1's queries here.

@terminator1 - one thing I'm curious about is how you came to believe that the recommendations you were making were necessary? Was it just from reading (but not trying to execute) the information in the IPython Notebook? Were you reading the code at face value and sort of introducing "grammatical fixes" into it based on English grammar? Just interested here since others may have similar questions.

I'm also curious if you read about the virtual machine experience but just chose not to try it, and why? It's a sincere and genuine question, because the virtual machine was designed to make the entire experience as easy as possible for folks without much of a programming background, and I want to make it as un-intimidating and accessible as possible. Not sure if I'm doing the best job at that just yet and would love to hear your thoughts.

Final question - do you have access to the text of Mining the Social Web, 2nd Edition? If not, then I think I need to get you an Early Release copy to review. You're at a major disadvantage if you are trying to make the most of its source code without the context of the book itself.

I really appreciate you taking the time to check this out and your involvement. Please hang in there. I'm committed to helping you in any way that I can.

[ptwobrussell]

Feel free to comment further on this issue, but I believe it's safe to close it at this point from the standpoint of whether or not these issues need to be resolved.

[terminator1]

Hi Matthew,

Thank you for taking the time to respond to my query. I currently don't have access to the early release copy of the second edition, so I am at a disadvantage. My email address is nickmayne@gmail.com. Please send me the necessary credentials. I will certainly contribute to improving the experience as requested.

I'll respond to your questions piecemeal:

one thing I'm curious about is how you came to believe that the recommendations you were making were necessary? Was it just from reading (but not trying to execute) the information in the IPython Notebook? Were you reading the code at face value and sort of introducing "grammatical fixes" into it based on English grammar? Just interested here since others may have similar questions.

I opened the GitHub 2nd edition link you sent me and started looking through the folders etc. I saw that iPython was in the 1st edition github also, but I was able to make a fair amount of progress without going down the iPython path, so I just ignored it from the jump. Also the code in iPython was very unusual and unfamiliar (arrays with source code etc. in it). However, below all the folders was the chapters links, and in the chapters there was familiar code and explanations about getting the access token, making sure permission were set etc. So I just save the code to a file in the regular python GUI and started running it once I had everything from facebook set up.

In regards to the grammatical error issue, someone on the web mentioned that it ran successfully when you removed the friends.fields(likes) parameter. So I removed it and ran it and it worked! After that I started tinkering with that piece of code and eventually I got it to work with the singular word.

Please keep in mind the context. I was still working through 1st edition code and wanted to briefly run through the 2nd edition stuff since you mentioned it. That's why I was rushing through everything. However, I'm going to focus my energy on the 2nd edition code now because I'm literally stuck on the couchdb harvest timeline stuff. While I was able to successfully install couchdb, I get a ServerError 500 mismatch thing when db.view is used in the code. I've invested a lot of time trying to figure that out, but I think its time to move on. Also, in the 1st edition, there is use of sys.argv[1] whenever you expect something from the user. I'm running a windows environment, and I figure that is really for the Linux crowd. Sometimes it's difficult to know exactly what to replace them with. I'm not about to waste your valuable time opening a bunch of issues on the 1st edition. So...

I'm also curious if you read about the virtual machine experience but just chose not to try it, and why? It's a sincere and genuine question, because the virtual machine was designed to make the entire experience as easy as possible for folks without much of a programming background, and I want to make it as un-intimidating and accessible as possible. Not sure if I'm doing the best job at that just yet and would love to hear your thoughts.

With all that being said above, I just figure it's best to cut my loses, move on to the 2nd edition, run the virtual machine as mentioned by Jason and see what happens. I've installed the Oracle VM and it seems to be straightforward. Installed Vagrant, but need time to work through the "Getting Started" docs so I have some context etc. I really hope the virtual machine flow is a lot less painful, and I'll be sure to let you know of anything that is left ambiguous after reading the early release copy and suggested materials.

Final question - do you have access to the text of Mining the Social Web, 2nd Edition? If not, then I think I need to get you an Early Release copy to review. You're at a major disadvantage if you are trying to make the most of its source code without the context of the book itself.

As mentioned above, please send credentials to nickmayne@gmail.com, and I'll do my best.

SOME FINAL THOUGHTS I appreciate the time and dedication you display to helping people sort through the coding safari. The problem with programming in general is that there are too many technologies and connections to be made (between them) just to make the simplest of things to work. Not to mention the constant updating of software versions and changes in API calls (without proper communication) etc. Despite this you manage it pretty well, and GitHub is a great service.

In the future, I'm hoping that everything will be cloud based, and streamlined. I feel the virtual machine flow is a step in that direction. So for example, log into one site, check the boxes for the development environment you desire, drop the code in, press Go. All delivered from ONE vendor. Also known as vertical integration. A thing Steve Jobs pioneered, and Larry Ellison is betting all of Oracle on. lol.

Anyway, will wait to get the release copy and I'll send you feedback ASAP.

Thanks much!!!

[terminator1]

Hi Jason & Matthew,

Thank you for taking the time to respond to my issue. Have a problem at the very end..

I've successfully installed the Oracle Virtual Machine and Vagrant. I downloaded the files from the repository, navigated the command prompt to the vagrant directory and ran "vagrant up"

After that everything went fine except at the very end the Vagrant machine failed to boot from the prompt after downloading and processing everything. It advised me to try and run it from the Oracle Virtual machine to get a more elaborate explanation which was this:

result Code: E_FAIL (0x80004005) Component: Console Interface: IConsole {db7ab4ca-2a3f-4183-9243-c1208da92392}

I eventually researched it online and some suggested it was the number of CPU cores being requested in the settings. After some experimenting, it was the chipset setting that did the trick. It was at PIIX3 and I changed it to ICH9. Please note I'm running Windows 7 32-bit OS Intel Dual CPU.

An attempt to restart seemed to work, with a black screen in the middle and eventually an option to choose OS. There is Linux Ubuntu, Linux Ubuntu (recovery mode), and some other memory set options. (I was supposed to install the Linux VM right?!). After making the Linux Ubuntu choice, the black screen resumes and nothing else happens.

I also tried navigating to http://localhost:8888 after the "vagrant up" processing, but the page doesn't resolve to anything. Even after successfully starting the VM the page doesn't resolve to anything. I figure it has something to do with the initial boot up failing, which derailed everything that was supposed to happen.

Just for my frame of reference, iPython notebook is supposed to work in the browser right. The Oracle VM is just there to manage the processing etc. But iPython should run from http://localhost:8888

Thanks again guys!

[terminator1]

So I have a laptop at work running Windows 7 64-bit Intel i3 processor. Followed the same flow yesterday with it and had similar problems. But upon starting it up today, everything seems to work (Didn't even have to change chipset or anything like that). My work computer is my new best friend!

Logged into the Ubuntu using "vagrant" as username and password. Got the "Welcome to Vagrant-built virtual machine"

Now just trying to figure out what to do next.

[ptwobrussell]

Interesting. It does sound like there's just some general learning curve here as well as some quirks with perhaps how all of this works on Windows. Is there a reason you didn't use "vagrant ssh" to login to the system and chose to login to the machine with a username and password? The whole idea with vagrant is that it provides a layer of abstraction on top of Virtualbox (and other virtualization providers) for you so that you don't have to deal with some of these things. e.g. you would "vagrant up" and then "vagrant ssh" into the machine.

[terminator1]

Oh I didn't know about all that. Is it outlined somewhere? It's not in the book or Vagrant "Getting Started" documentation.

I just tried to start the virtual machine this morning from Oracle VM. It worked and then it was asking me for a login and password. So I just searched the internet and it said to use vagrant as login and password.

Right now in the VM window I have a Linux command prompt.

So should I shut down the virtual machine from Oracle VM and try to vagrant up from the directory and login using "vagrant ssh"? I don't want to go down a path you guys aren't familiar with.

[ptwobrussell]

Which docs are you looking at? The ones linked from the README are to this site, though perhaps not this exact page - http://docs.vagrantup.com/v2/getting-started/up.html - check out that general class of instructions in the getting started guide and let me know if it makes sense.

I am thinking that a 2 minute YouTube video that shows how all of this works might be helpful. For some reason, it seems as though the bar to entry might still be a little too high. What do you think?

[terminator1]

Ok I wasn't looking at those docs.

Yes a 2-minute YouTube video would be helpful. I also think a PDF doc with screenshots would be helpful also. I can help you build a draft for the Windows 7 flow with screenshots and all.

The two main problems I'm having is that there isn't one place that addresses the entire flow from start to finish of what we are trying to achieve. There is mention of Oracle VM and associated docs, then there is mention of Vagrant and associated docs, then there is iPython which is supposed to run in the browser. As a user I'm caught in the middle and anxious about how the three worlds will connect after each is installed.

I'm curious. How many of your readers do you anticipate will be Windows-based vs. Linux based?

For Windows I know that there will be issues with the chipset the VM is running on. From the internet I gather that people running the VM on a desktop machine generally has more problems than people running on a laptop. The number of CPU cores may also introduce problems This is certainly true in my case. But we'll take it one step at a time.

[terminator1]

I did the "vagrant ssh" command. It asks me if I have an SSH client installed and makes some recommendations.

My only real problem now is that http://localhost:8888 isn't working as Jason mentioned above.

http://localhost:2222/ gives the following message in my browser however:

SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1 Protocol mismatch.

[terminator1]

Is there something I can do to get iPython running from the Linux command prompt in Oracle VM?

[ptwobrussell]

I would recommend doing a "vagrant halt" followed by a fresh "vagrant up". I know that @jwsy tested this yesterday on a Windows 7 box and it all worked as expected, so I'm curious if something went wrong on your first attempt because of the config problem you were having, but would not surface in a second attempt. The hope is that you can access localhost:8888 in your browser once the VM starts.

If you can't, then, we'll need to login to the box using SSH as a way of looking at its log to see what went wrong. You will need to install and learn about PuTTY, so that when you type "vagrant ssh", it has a way of logging you into the machine as the first step in overcoming that - and to be honest, it's very worth your time to learn how to do that if you are interested in software development.

I am currently working on getting a Windows machine configured and hope to be able to try and be of more help, but it's going to take me a few days to get it all sorted out. Once I do, I will better understanding the Windows specific issues and be able to update the docs to make mention of installing PuTTY and probably also include a screencast for Windows users specifically.

[terminator1]

My intuition is that its a problem with the network that is preventing localhost:8888 from showing up on my machine. Will put some energy into installing putty and see what happens. Will keep you posted.

[terminator1]

Is there a special directory that the vagrant files from the repository should be placed? Or it doesn't matter?

I cd to the directory that my vagrant files are in, then vagrant up. The files are located in a folder on my desktop.

[jwsy]

@terminator1 can you try a vagrant destroy and then vagrant up again? I've actually had some problems downloading python packages recently (I wasn't able to download Flask this morning) that could have done the same things.

[terminator1]

@ptwobrussell I got PuTTY running and successfully connected to the VM. I now have a Linux command prompt in front of me, which I consider to be huge progress to say the least.

I also installed curl, cause I've seen it a bunch of times when trying to troubleshoot this issue.

May you suggest some commands to run so that I can get an idea of whats going on. Again, the only real issue is that localhost:8888 isn't showing up in my browser. I even tried changing the port to localhost:3333 to see if it would help, but same behavior.

localhost:2222 shows up fine in the browser.

I'll copy and paste the output from your commands here.

@jwsy tried what you suggested, but nothing changed.

Thanks!

[ptwobrussell]

@terminator1 - I just updated the repo with what I believe is a much better shared folder setup, based largely on this thread. If you could "vagrant destroy" and then "vagrant up", you'll find that once you're up and running again that everything that happens on the Vagrant box (such as data files that it writes out or IPython Notebook modifications) are kept in synchronization with the files on your host machine. The bottom line is that this should prevent users from having to ever think about using SSH to login to that Vagrant box with a terminal unless there is some kind of specialized configuration problem, which hopefully won't be the case very often once we get all of the kinks worked out.

@jwsy - I think you might like this as well. It completely blurs the line between the guest and the host files now. See bootstrap.sh for the details. Haven't attempted to use virtualenv yet to install the entire Python environment in the shared folder, but the basic makings are in the works now.

[terminator1]

Will test on home and work computer and report what happens.

[terminator1]

@ptwobrussell I ran the new repo on my home computer last night (5/29/13) and it didn't boot up on the initial run. Basically identical issues as before. VT-x error etc.

Ran new repo on work computer this evening (5/30/13) and things went smoothly in terms of initial boot up. A whole lot of activity happened after "vagrant up". Unfortunately, localhost:8888 still not working. At the very end of the text pushed out on the command prompt after "vagrant up", there were some errors related to missing packages (URLError because name or service not found). I will email you an image of what it said. The VM continued to boot up fine though, so I guess its a step in the right direction.

I will work on the home computer a little more when I get home. I have a feeling there is a difference in behavior due to 64-bit OS (work) and 32-bit OS (home). Will keep you posted.

[ptwobrussell]

For reference, you can also just upload the images here on GitHub by dragging them onto the text area. I'll include yours here for completeness. At this point, it looks like we have something concrete to work with, which is good.

Over the weekend, I should have a Windows 7 system setup, so I should be able to start testing a bit.

Can you type the full URL shown in the error screenshot, the one that is of the form https://downloads.sourceforge.net.../matplotlib-1.2.1.tar.gz and see if you can download it from your computer?

Or try pulling it down with curl on the command line in the VM would be even better. e.g.

curl https://downloads.....

It's curious that you are getting SSL errors like that.

Just to clarify - you are running Windows 7, correct? (Not Windows XP or 8?)

And where are you located? U.S.? England? Trying to figure out if that has anything at all to do with it. One other fellow I was speaking with cannot "pip install geopy", and he lives in Canada, and he was having a similar kind of (but not the exact same) http error.

[ptwobrussell]

Can you also tell me what version of pip you are running?

pip --version

[terminator1]

I'm actually in Jamaica. Running Windows 7 (64 bit edition)

Running pip 1.0.1

Will try your suggestion.

[ptwobrussell]

@terminator1 - Just successfully got Windows 7 (32 bit edition) installed with Bootcamp, so tomorrow I hope to go through the steps to see if I can reproduce your latest issue in the screenshot. It will be nice to finally have an environment that matches your system so that I can better support you and other Windows users. Through this experience, I have really come to a better appreciation of how to hopefully make the experience as enjoyable as possible for folks who don't come from a world involving terminals and Linux.

I'd be curious on your thoughts to the latest notes I've added to the Chapter0 notebook if you get a chance to review it. What makes sense and what doesn't make sense with regard to running commands and such on the virtual machine as I've outlined it?

[ptwobrussell]

@terminator1 - I went ahead and just went through what I think would be a standard Windows install, and I think I may have identified at least one of your problems. Here's what I did

• Installed Windows 7 (32 bit)
• Installed lastest version of Virtualbox
• Installed latest version of Vagrant
• Installed latest version of Git from http://msysgit.github.io/ (because it comes with a built-in ssh client. See screenshot below for an important configuration setting you need to choose when installing it.)
• Open up a new command prompt
• Type ssh.exe to verify that you see output from this utility that would come bundled with Git
• Type vagrant init to create an empty Vagrant file
• Adjust your Vagrant file to have these two settings: * config.vm.box = "precise32" * config.vm.box_url = "http://files.vagrantup.com/precise32.box"
• Type "vagrant up" \ The box came up as expected
• Type "vagrant ssh" \ I logged into the box automatically as hoped/expected (thanks to ssh.exe being on the system)

I can't remember the specifics, but you mentioned that the code worked on your work computer (a 64 bit machine) but not on your home computer (a 32 bit machine), correct?

Well, as obvious as it now seems, here's what I think could have been happening - the Vagrantfile for Mining the Social Web is set to use the precise64 base image, which is an image intended for a 64-bit system, so when you tried to run it on your system at home, the 64-bit virtual machine vs 32-bit host system just couldn't work together, which makes sense once you realize it because they're entirely different architectures.

I think the easy solution for you, if you can reproduce those steps above, is to simply edit the Vagrantfile in your checkout to be the same precise32 image that you would have just downloaded. From there, if you "vagrant destroy" and "vagrant up" again, I have a lot of confidence that you will finally have joy.

This doesn't necessarily sort out that last screenshot you sent me, but I think it gets back to the original problem we started out on. Let me know what you think/find out as you try it for yourself.

It has been so helpful having you to work with through all of this. I really owe you for pushing me in this direction, because it's been an incredibly valuable journey that has definitely shaped things for the better. I will be updating the docs and config (somehow) to try and alert users that they will need to use the proper images, or better yet, try to figure out if there's a way to automatically detect if they have a 32-bit of 64-bit architecture and just automatically handle it for them. I know this can be done on Mac/Linux, but I'm not sure about Windows. There's probably a way, but I just don't know how to do it yet.

As I finished writing this up, my VM finished configuring my Mining the Social Web VM, and everything seems ok so far. Haven't fully tested yet, but it's looking good at a glance.

A few screenshots for you: Pick this option when installing Git (and notice the other stuff in the command prompt)

A successful "vagrant ssh" into a precise32 box

What appears to be a successful Mining the Social Web install with IPython Notebook running (it hasn't been fully tested yet but is probably fine)

CC: @jwsy

[terminator1]

@ptwobrussell BINGO!!!

Everything worked like a charm on the 32-bit Windows 7 at home.

Updated the Vagrant file to 32 as you mentioned.

Was able to bring up localhost:8888 finally.

NOTE: Box 'precise32' was not found, so it downloaded it from the URL provided. Not sure if it will be a concern to you, but either way it worked.

Will test on the work computer tomorrow.

Thank you also for having the patience with me. You really are committed! I will give you feedback on the Chapter now that I have everything running.

I suggest you include the screenshots in the book as you did here. It is really helpful so that people know what to expect etc.

Will be in touch. Thanks much!!

[ptwobrussell]

Boomshakalaka!

Btw, it is expected that the precise32 box would need to be downloaded, so that's as expected.

[terminator1]

Something to consider:
I know that when I installed and ran WAMP server several years back that I would not be able to connect to the localhost because I was logged into Skype. I'm not sure if the same issue would apply to locahost:8888 as it seems to be an unusual port number that other programs probably won't be using. But just to make life easier for everyone and to avoid a potential hiccup for users, you might want to mention that you don't want any other Apache based services running that might clash with it. For example, CouchDB from the 1st edition automatically starts up and runs as a Windows service. It also uses Apache as its web server. Could it potentially create conflicts/problems? Same with WAMP. Maybe a Google search of localhost:8888 to see other programs that could be running on the host machine and conflict with iPython notebook. Just trying to make the process as robust and scaleable as possible.

Initial thoughts on Chapter 0:

Overall the Chapter is well-written, and I'm just being very picky but I guess its good for you to know:

When you download the repository, there are two README files that are included.

• Lets assume we downloaded the repository into C:\Domayne\Social
• There is a README file in C:\Domayne\Social
• There is a README file in C:\Domayne\Social\vagrant When I ran the Hello.txt script, I assumed it would go to C:\Test\vagrant because that is where the Vagrantfile is and where I navigated the command prompt to in order to "vagrant up". I eventually realized it was in C:\Domayne\Social because the script listed out the files contained in the folder and I knew where url_download.jpg file was. You mention that the folder with the README file is where everything is synced. Is it possible that you can rename the file in C:\Domayne\Social\vagrant to HELP so as not to confuse people and throw them off. Not a big deal though.

That's it really. This iPython Notebook thing is truly beautiful though from what I can see so far. I'm sure many more people will get more value from the book and source code because of it.

[ptwobrussell]

Thank you. I appreciate this!

Lets open new issues for suggestions on other notebooks since I think we have long since dealt with this ticket's original issue :)