Can't use automatic variables in before statement

[nickodell]

I'm using pyp to parse and munge CSV/TSV files.

Recently, I ran into an issue where I can't use stdin in a before statement.

Here's my code, in Bash:

< pollution.csv python3 pyp.py \
    -b 'input = csv.DictReader(stdin, delimiter=",")' \
    -b 'output = csv.DictWriter(sys.stdout, fieldnames=input.fieldnames)' \
    -b 'output.writeheader()' \
    'for row in input: output.writerow(row); pass'

(This is just a dummy command which doesn't modify the input.)

Here's the error:

Possible reconstructed traceback (most recent call last):
  File "<pyp>", in <module>
    input = csv.DictReader(stdin, delimiter=',')
NameError: name 'stdin' is not defined

Looking at the error, it looks like the code assumes that an automatic variable (eg. lines, x, stdin) will not be used during a before statement. For some of them, this is reasonable, because they don't have a meaning outside of a loop. But lines and stdin have perfectly sensible meanings before the loop.

[hauntsaninja]

Thanks again for your earlier PRs!

My intent with the before flag wasn't about loops as much as "before processing input", which is the phrasing used in pyp --help and the README.

Anyway, the idea here is that you want to process input with stdin, so just passing code in without any flags should work.

~ λ pyp --explain 'input = csv.DictReader(stdin, delimiter=",")' 'output = csv.DictWriter(sys.stdout, fieldnames=input.fieldnames)' 'output.writeheader()' 'for row in input: output.writerow(row); pass'
#!/usr/bin/env python3

import csv
import sys
stdin = sys.stdin
input = csv.DictReader(stdin, delimiter=',')
output = csv.DictWriter(sys.stdout, fieldnames=input.fieldnames)
output.writeheader()
for row in input:
    output.writerow(row)

It's true that magically introduced loops are what really make the difference between before code, main code and after code useful / meaningful, which I guess makes things susceptible to confusion in their absence. I'll see if I can think of ways to document things more clearly. Removing the difference between before, main and after code when using lines and stdin would be tricky implementation-wise, and might in fact cause more confusion since the flags would then do nothing.

By the way, in case you missed it, I recently added the ability to add some configuration to pyp: https://github.com/hauntsaninja/pyp#pyp-is-configurable. So you can now, e.g, define a print_csv function that you could then use :-)

[hauntsaninja]

I special-cased this error, so pyp should be a little bit more helpful if people run into this :-)

[nickodell]

@hauntsaninja Thank you for adding a helpful error message. To me, that's just as good as what I requested.