Working directory for testing?

[k8hertweck]

I've put on my freshest eyes as I've been stumbling through the run though for #560 . The more I proceed, the more I realize how desperately I would want to have Amira's repo as a reference. Understanding where pieces of code go in each file is really really useful, and I've got commits for each chapter/section (that can also be tagged as such).

The biggest stumbling block has been reconciling changes following reorganization of chapters. Usually this is manageable, but I could use some advice for the testing chapter. It's currently set up with zipf as the project directory, but we can't directly import our Zipf's functions (e.g., import countwords) in this way. I think before this was after the packaging chapter so it was more straightforward?

Anyway, I think we have two options:

1. make bin the working directory for the testing chapter, and then move files (test_data) around later on (this will also require adjusting paths to data in at least one place).
2. Use a method to allow importing scripts from a different directory. Anyone have a favorite way to do this?

[DamienIrving]

The first thing to say is that running pytest from the ~/zipf directory works fine, because test_zipfs.py is located in ~/zipf/bin along with the scripts it imports (countwords.py etc) and according to the definitive guide to python import statements:

• import statements search through the list of paths in sys.path
• sys.path always includes the path of the script invoked on the command line and is agnostic to the working directory on the command line

So the only issue is the following code block in the beginning of the chapter, when we are running things interactively (rather than invoking test_zipfs.py via pytest):

import countwords

with open('test_data/risk.txt', 'r') as reader:
    actual_result = countwords.count_words(reader)
assert actual_result == expected_result

Option 1 - from bin import countwords

This won't work because countwords.py imports other scripts...

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/amira/zipf/bin/countwords.py", line 10, in <module>
    import utilities as util
ModuleNotFoundError: No module named 'utilities'

Option 2 - update sys.path

import sys
sys.path.append('/Users/amira/zipf/bin')
import countwords

Option 3 - work from the ~/zipf/bin directory

$ cd ~/zipf/bin

I think it would be worth doing option 2 and having a call-out box explaining why we had to do sys.path.append (and that the other option would be $ cd ~/zipf/bin). The box could link to the definitive guide for more information. People often have trouble importing libraries (even experienced Python users), so I think this would be a good addition to the book.

[k8hertweck]

Thanks so much for the well-reasoned and thorough explanation; I am comforted by the knowledge that this is a common stumbling block and my brain isn't simply turning to mush from trying to consume a book's worth of knowledge in a few days.

I'll include these changes in #560

[k8hertweck]

resolved by d78fd0e2