Python Text Analysis Fundamentals: Parts 1-3

Overview

This workshop will equip newcowers with the foundation for applying computational text analysis methods in their work. The focus is on high-level descriptions of what existing methods do and user-friendly implementations. We will also spend some time on interpreting results correctly.

What you'll learn

• what computational text analysis can do, and what it can't do

• preprocessing text data

• implementing simple unsupervised methods (tf-idf, topic model, cosine similarity) and supervised methods (classification with logistic regression) using a bag-of-word approach

• interpreting results

• if time permits, introduction to word vector representations

• where to go next to learn and seek help with your computational text analysis projects

Prerequisites

We will get our hands dirty implementing some of the methods. This will be in Python. If you would like to follow along with the implementation details, you will need some familiarity with Python. Completion of D-Lab's Python FUN!damentals workshop series is sufficient. If you haven't programmed in Python or at all, you are of course welcome to attend and learn the big ideas behind the methods.

Getting started & software prerequisites

Anaconda

If you want to run the code on your computer, you have two options. Both involve installing (or making sure you have installed) Python 3 and some additional libraries. Anaconda is a free product that makes the installation process easy. It bundles together the Python language and a whole bunch of additional packages that we often rely on in our workshops. This way, you only have to download and install one thing. To use this method, visit this site and follow the instructions for your operating system to download the Python 3.x version (it might be 3.6, or 3.7, or higher). Please, please, please download the 3.x version, not the Python 2.x version. You may have a choice between using the graphical installer or the command line installer. Use whichever you're comfortable with, but the graphical one is easier.

For Pythonistas

If you've been using Python for a while, you might not want to use Anaconda. First, make sure you have a Python 3.x version. Second, install Jupyter by following these instructions. Third, install the packages listed in the requirements.txt file of this repo. The easiest way to do this is:

pip3 install -r requirements.txt

Are Anaconda, Python, or Jupyter not working on your laptop?

You can also access the workshop materials through your browser on UC Berkeley's DataHub by clicking this link. Datahub is a great option if you aren't able to install Anaconda, Python, or Jupyter locally. CalNet ID credentials required.

IOKN2K

It's OK Not To Know! That's our motto at D-Lab. D-Lab is open to researchers and professionals from all disciplines and levels of experience.

Resources

• CTAWG (Computational Text Analsysis Working Group) website
• Lectures from Stanford's NLP class
• Info 256 - Applied NLP class by David Bamman

Contributing

If you spot a problem with these materials, please make an issue describing the problem.

Acknowledgments

• Ben Gebre-Medhin
• Laura Nelson
• Teddy Roland
• Geoff Bacon
• Caroline Le Pennec-Caldichoury

These materials have evolved over a number of years. They were first developed for the D-Lab by Laura Nelson & Teddy Roland, with contributions and revisions made by Ben Gebre-Medhin, Geoff Bacon and most recently updated by Caroline Le Pennec-Caldichoury.