J-Term materials: abstracts, software installation, data, etc

[cranndarach]

I've started a document where we might want to keep the most up-to-date info, while we discuss it here, if that works? software-and-instructions.md

Here's what I have so far:

Titles & Abstracts

Introduction to Python

[Abstract TBA]

[Monica's title TBA]

[Abstract TBA]

[Russel's title TBA]

[Abstract TBA]

Software to download

• Anaconda Python 3.5 (RS, RR)
• Psychopy (ML)
• A text editor / IDE such as Atom, Sublime, Gedit, Spyder (included with Anaconda), PyCharm, Emacs, or Vim, not a word processor - (RS)

[cranndarach]

RE Russel's concerns about multiple Python distributions: I don't think the Psychopy standalone will conflict with anything else, because it should be self-contained. That said, I had trouble getting it working on both Windows (where it just never made any indication of thinking about starting...) and Linux Mint using the Neurodebian package iirc (where it segfaulted).

So I wonder if problems like that are going to be common, and if it would be easier to suggest that people install the library (it's on Conda) and set up a virtual environment with Anaconda in Python 2.7 for Psychopy, which is way easier than it sounds (conda create -n psychopy-tutorial python-2.7 psychopy should do it). It would also have the benefit of not asking people to install more software (at least, no downloading and GUI installing), because they can probably use the same editor for both Monica's and mine. But I do get that having to use a terminal might scare people off, so it might be more like six of one, half dozen of the other.

[cranndarach]

Edited original post with suggested text editors — comments welcome!

[drussellmrichie]

Thanks so much for doing all this, Rachael! This is a huge organizational help. If this series of workshops goes well, I hope you, Monica, and/or others can reuse them in future J-Terms or other IGERT/NBL/IBACS training events! On to the issues:

The software list you generated sounds generally fine, although I think we'll probably want to make a recommendation of a particular text editor that will be friendliest to novices. I trust your judgment on this -- I do most of my programming not in text editors but in IDE's like Rodeo or jupyter notebooks.

So I just tried typing conda create -n psychopy-tutorial python-2.7 psychopy at my terminal, and I got this:

Fetching package metadata .........
Solving package specifications: .

PackageNotFoundError: Package not found: '' Packages missing in current osx-64 channels: 
  - python-2.7
  - psychopy

Close matches found; did you mean one of these?

    psychopy: psycopg2`zx
    python-2.7: python, ipython, python.app

You can search for packages on anaconda.org with

    anaconda search -t conda python-2.7

(and similarly for the other packages)

I had to do something slightly different, partly following this discussion on the PsychoPy boards:

conda create -n psychopy-tutorial python=2.7
source activate psychopy-tutorial
conda install wxpython
pip install pyglet psychopy

That 'works', although I haven't actually tried to run a psychopy script yet.

I wonder if @monicaycli has opinions about these things.

[cranndarach]

Whoops, I meant python=2.7, sorry about that.

This works for me now:

conda create -n psychopy-tutorial --channel https://conda.anaconda.org/CogSci psychopy pyglet wxpython python=2.7

And then apparently on Windows it's just activate [env-name] instead of source activate [env-name], and likewise deactivate instead of source deactivate.

Explanation of how I got there:

Neither psychopy nor pyglet are in the main conda channel, but running:

anaconda search -t conda psychopy

yields the following:

Using Anaconda Cloud api site https://api.anaconda.org
Run 'anaconda show <USER/PACKAGE>' to get more details:
Packages:
     Name                      |  Version | Package Types   | Platforms
     ------------------------- |   ------ | --------------- | ---------------
     CogSci/psychopy           |  1.82.01 | conda           | linux-64, win-32, win-64, osx-64
                                          : Psychophysics toolkit for Python
     erik/psychopy             | 1.80.01dev | conda           | linux-64, osx-64
                                          : For running psychology and neuroscience experiments
     jcarlin/psychopy          | 1.80.01dev | conda           | osx-64
                                          : For running psychology and neuroscience experiments
     kschuler/psychopy         |  1.82.01 | conda           | osx-64
                                          : Psychophysics toolkit for Python
Found 4 packages

The first entry looks promising: CogSci sounds legit and has a version for every relevant OS. So next I checked for what channel to use to get it:

anaconda show CogSci/psychopy

which yielded:

Using Anaconda Cloud api site https://api.anaconda.org
Name:    psychopy
Summary: Psychophysics toolkit for Python
Access:  public
Package Types:  conda
Versions:
   + 1.82.01

To install this package with conda run:
     conda install --channel https://conda.anaconda.org/CogSci psychopy

But when creating an environment, you can specify packages to install to it rather than running a separate conda install command. You'd just need to specify the --channel in the same way as in the example above. That tells conda to essentially append the channel to its path-equivalent.

And the same thing goes for pyglet — CogSci has a version too, so the channel only needs to be added to the path once. If you search for wxpython, you'll see an entry from anaconda, so it should find it by default.

So that leaves the steps: 1) The conda create call, 2) name the env with -n psychopy-tutorial, 3) add the CogSci channel to the search path with --channel https://conda.anaconda.org/CogSci, and finally 4) specify what packages, including python, it should install in the new environment.

[cranndarach]

Wrt text editors, maybe:

Your favorite code-oriented text editor (not a word processor), preferably with Markdown support. If you don't have a favorite, I recommend Atom (https://atom.io).

Not sure how to eloquentize "code-oriented text editor" though. "Text editor" could lead to people bringing notepad and never coding again, but "code editor" isn't really what they're called. Maybe "text editor with syntax highlighting" would work, but the term "syntax highlighting" isn't super transparent.

[drussellmrichie]

Okay, I used that new line you generated, activated the environment, downloaded one of @monicaycli's psychopy scripts, and ran it with python 01_text.py. It seemed to work -- it opened a PsychoPy window and displayed "Hello World".

So it seems like this process should work, although I expect we'll eventually want to test all of the PsychoPy scripts Monica plans to use in this environment? I think this puts us in pretty good shape as far as sending installation instructions to Yanina and the prospective workshop attendees?

As for the precise language wrt text editors, I wouldn't sweat that detail too much. I think what you wrote is fine. Most people will just do what you recommend.

[drussellmrichie]

Okay, since Yanina's email indicated she wanted to send titles, abstracts, and software installation instructions by Xmas, and wanted things from session leaders by Dec 20-22, I think we should nail down all the installation-related things for our sessions in the next day or so. To that end, I would like to suggest the below addition to @cranndarach's "Software to download" section above. It looks like a lot, but....I don't really know a way around that? @monicaycli, I realize you are probably pretty busy with wrapping up end-of-the-semester stuff, but it'd be great if you could just check out the portion(s) relevant to you.

The instructions below only account for Windows and Mac, because (a) I bet only 1 or 2 people use Linux, (2) I have no idea how Linux works, and (3) Linux users can probably figure things out for themselves. :-P

Installation and setup for Python sessions

1. Anaconda Python 3.5, which contains the Jupyter Notebook which Russell's session will use.
   1. Follow this link, and then click the icon for your computer's OS. You want Python 3.5. For Mac, you will probably find it easier to use the graphical installer. (For Windows, it appears you can only install via graphical installer!)
   2. Once you install Anaconda, if you open a terminal (Mac) or cmd prompt (Windows) and type python --version***, the terminal/cmd-prompt should you should see 'Anaconda' somewhere in the output.
   3. If you type jupyter notebook from the terminal/cmd-prompt, a tab with a file explorer interface should open in your default web browser. If you click 'New' in the upper-right, you should see an option for 'Notebook -- Python [root]'. If you click that, a new jupyter notebook should open in a new tab. Russell's session will be conducted in one of these.
2. A code-oriented text editor / IDE (text editor != word processor), preferably with Markdown support, such as Atom (Rachael's recommendation), Sublime, Gedit, Spyder (included with Anaconda), PyCharm, Emacs, or Vim. You will use a text editor / IDE for Rachael's portion.
3. PsychoPy. To install PsychoPy, first install the Anaconda distribution of Python in step 1, and then do the following:
   1. Open a terminal (Mac) or cmd prompt (Windows) and type conda create -n psychopy-tutorial --channel https://conda.anaconda.org/CogSci psychopy pyglet wxpython python=2.7. That should be all one line. This will (1) create a Python 2.7 'environment' on your machine, and (2) install PsychoPy and its dependencies (required packages/libraries) to that environment.
   2. To activate this new environment when you want to work with PsychoPy, type source activate psychopy-tutorial if you are on a Mac and activate psychopy-tutorial if you are on Windows.
   3. You should now be able to run a psychopy script in the current working directory with python <psychopy-filename.py> (replacing <psychopy-filename.py> with the actual filename!).
   4. To deactivate this environment and return to the default Python 3.5 environment installed by Anaconda (so you can work on Rachael's and Russell's sessions, for example), type source deactivate on Mac and deactivate on Windows.

*Does opening a cmd prompt on Windows typing python --version work?**

I'm also not sure that this list of instructions is quite complete, because it doesn't say anything about navigating directories via the command line. I'm not sure people will need to do that for my portion -- they might be able to do that via the graphical file explorer in the jupyter notebook server -- but maybe they will for Monica's PsychoPy session? I guess it depends on how people will be running PsychoPy scripts (in the command line vs in an IDE?).

Suggestions/edits welcome.

[cranndarach]

Looks good, though I think telling people to type python --version might be simpler and faster than asking them to start the interpreter, because this just prints the version and exits.

rachael@rachael-HP8470p ~ $ python --version
Python 3.5.2 :: Anaconda 4.2.0 (64-bit)

I think everything you said for Mac also works for Linux (at least, Debian-based systems, but *NIX systems are usually pretty similar).

I feel like it might be complicated to explain how to cd in the installation instructions. Might be better to plan to be able to handle issues in the actual sessions?

[drussellmrichie]

Ah, yes, just having them type python --version is a good suggestion. I just edited my previous comment accordingly.

I think your call about explaining -- or rather not explaining -- how to cd around in these instructions is reasonable. I don't expect it will affect my portion as much, so I largely leave it up to you and Monica.

[cranndarach]

Now that I'm thinking about it, people who already had Python installed (particularly Mac users) might have to edit their path or .bashrc / shell profile to make sure python points to the Anaconda distribution. Checking for the version is actually a way to test for this, because if it doesn't say "Anaconda," or if it has the wrong version of Python, then that's a problem.

On Windows, it can be a nightmare that involves installing py.exe, the Python version picker, which does not come with Anaconda, and learning to write Unix-like shebang lines, so hopefully whoever is using Windows doesn't already have another Python version/distribution installed.

For Unix-like systems, if it's wrong, an alias can be added so that typing python points to the Anaconda distribution:

which anaconda

Should output something like /home/[USER]/anaconda3/bin/anaconda. Then:

nano ~/.bashrc

(I think Nano is part of Bash, but of course anyone who has a preferred editor can use their own.)

Add the following:

alias python='/home/[USER]/anaconda3/python'

where the path is everything that came before /bin/anaconda when searching for which anaconda, plus python.

Then to quit the editor, press Ctrl+x, y, return.

Alternatively

We could ask people who have this problem to simply create a Conda environment for the tutorials, which will point to the correct python.

Or, this only just occurred to me, and I can't decide if it's easier or if it will just cause more problems, but we could just ask people to call python3 instead of python. But this might still not work on Windows iirc.

[drussellmrichie]

Hmmm, here's where I'm a little out of my depth.

I think for Windows users, there's a good chance none of them have installed Python already, and if anyone has, (a) it might be the Anaconda distribution, and (b) if not, they may be savvy enough to figure things out on their own / with not much help from us?

As for Mac and other Unix-like systems, I'm not sure what the best solution is. I think I'm a little disinclined to setting up a Conda environment just for the tutorials, because that seems like it will be of limited use to people after J-Term? I think the solution should be something that they can "Set it, and forget it". That is, something they do once which lets them use the Anaconda Python 3 distribution moving forward without having to think hard about it. Managing environments is, IMO, one of the worst aspects of programming, and we don't want people to have to continue to deal with it after J-Term because I think that will be a barrier to people continuing to use and learn Python.

Is this another possible way forward: we just mention in the instructions that if someone types python --version and they don't get anaconda in the output, they can let us know, and we'll post something somewhere (on github? the IGERT listserv?)? The instructions we have above are already long and intimidating, and I think we want to minimize them?

I don't know. I think you are more knowledgeable about this sort of thing, @cranndarach, so I'm also inclined to trust your judgment!

[cranndarach]

I agree that making people think about environments is likely to create barriers, and that too many instructions can be intimidating. Having people contact us if it's pointing to the wrong place is probably the best course of action—I'd be happy to field those questions as much as I can.

[drussellmrichie]

Okay. Sounds like we're in agreement then. I'll edit the Installation instructions step 1.2 with a note asking people to contact us if python --version is pointing to the wrong place.

I just thought of another little thing to add: a sample PsychoPy file that people can run to test their PsychoPy installation ahead of @monicaycli's workshop. I'll just take Monica's Hello World example and put it in the repo for now -- she can feel free to remove it later when she puts her materials in their own subdirectory.

With those adjustments, do you feel that these instructions are complete enough to have Yanina link to them in the J-term program? I guess the nice thing about linking to the markdown file in the repo -- rather than copying and pasting the instructions into the program -- is that we can continue to tweak them if need be.

[cranndarach]

I think it looks good. And I'll update with an abstract shortly.

[cranndarach]

Abstract is up as of a93c2ac!

[drussellmrichie]

So sorry to keep bugging @cranndarach and @monicaycli about the python workshops, but another thing occurred to me. I imagine that, at some point, we'll want people to clone or, more likely (since it doesn't require forcing people to use git), just download the entire repo so they have our data, code, etc. However, I'm still developing my data analysis materials (I expect to finish in the next couple days), and I suspect Monica is as well?

So, (1) do you both agree we should add something in the instructions about downloading the entire repo, and (2) if so, it's probably best to figure out when we have truly finalized workshop materials, tell people to download that version of the repo, and then stop making major changes to the tutorial materials, yes?

[cranndarach]

I was originally assuming we'd each send out the relevant materials for our own sessions in an email or something. I'm not sure which is better between that and having people download the repo; I feel like either one could lead to either convenience or confusion.

[monicaycli]

I think having people downloading the entire repo is a good idea - it’s just the most straightforward way in my opinion, since all the files are already uploaded there. I agree that we should probably finalize and stop updating our materials soon so that everybody is on the same page. I actually just finished up my tutorial and pushed it to the repo. :)

Have a nice weekend, and I look forward to seeing y'all next week!

ps. sorry that I haven't really contributed very much to this thread...

[drussellmrichie]

Thanks, Monica!

I think I'm getting close to finishing mine (I'll push to the repo in a sec) -- I've covered quite a bit of Pandas and some Seaborn, and now need to try to fit some statsmodels and/or scipy.stats in there. I think I can finish over the weekend.

[drussellmrichie]

Just an update -- I think my jupyter notebook now covers more or less everything I want / can possibly do in about 1.5 hours (I figured I shouldn't plan to take up the whole 2 hours so we can troubleshoot here and there if necessary). Tomorrow, I'll go through it one more time, time myself, and then push to the repo. At that point, my stuff should be ready to be sent out to the entire community.

How are you two doing with finalizing your stuff?

[cranndarach]

I think mine is finalized now. I'm sure I'll find the occasional typo, but content-wise, I think it's ready to distribute. (Sorry, I intended to have it up yesterday but then the internet was down in my neighborhood.)

[cranndarach]

So I got an email from Yanina asking tomorrow's presenters to add our materials to the Drive folder, so I did that for mine. I forgot that that would be a thing, but it seems like it's now the more reasonable option.

I just uploaded all of the *.md files to "J-Term 2017/J-Term 2017 Resources/intro-to-python", not in a zip or anything — hopefully people will download the entire folder and that's one less step.

I hope it's okay to have changed things up according to new instructions, but let me know if there's any issue.

Technically, it is possible to upload a git repo to Drive, in case that's preferred for any reason. But then people would probably be downloading the repo, which may or may not be what we want.

[drussellmrichie]

I also just a minute ago pushed to the repo what I expect and hope will be the final version of my data analysis jupyter notebook.

Re: uploading to Google Drive, I guess you basically uploaded your whole intro-to-python folder to J-Term 2017 Resources? I guess @monicaycli and I could then do the same for our portions? I think that'll work, although I may need to then modify a couple file paths in my notebook (not a huge deal).

[cranndarach]

We could also keep the current repo structure by adding a folder to J-Term 2017 Resources/ for all three of our sessions, and having subfolders for each — whatever is easier.

[drussellmrichie]

Maybe we can discuss tomorrow at your workshop.