policybrain-builder CLI no longer working

[martinholmer]

@jbcrail, I'm having trouble releasing taxcalc 0.10.0 packages. Here's what is happening.

$ python setup.py install

$ conda list policybrain
# packages in environment at /Users/mrh/anaconda:
#
policybrain-builder       0.0.1                     <pip>

# cat taxcalc-package-cli.sh
#!/bin/bash
# Script to build and upload taxcalc package for specified release using
# policybrain-builder CLI, pb, with a single parameter, TAG (e.g., 0.9.0)

echo "STARTING : `date`"

TOKEN=<secret-token-that-is-good-well-into-2018>

pb release --token $TOKEN --label main taxcalc=$1

echo "FINISHED : `date`"
exit 0

$ ./taxcalc-package-cli.sh 0.10.0
STARTING : Mon Aug 28 16:38:55 EDT 2017
FINISHED : Mon Aug 28 16:38:59 EDT 2017

Why does pb seem to no longer work? It worked fine for me a couple of weeks ago. Am I doing something wrong?

[jbcrail]

Could you re-run the script but using verbose mode and post the results? For example,

pb release -vvv --token $TOKEN --label main taxcalc=$1

The extra -v flags increase the verbosity level.

[martinholmer]

@jbcrail said:

Could you re-run the script but using verbose mode and post the results?

Thanks for the quick response.

Here is what I got:

iMac2:policybrain-builder mrh$ ./taxcalc-package-cli.sh 0.10.0
STARTING : Mon Aug 28 19:06:37 EDT 2017
DEBUG: conda config --get always_yes
INFO: saved previous conda config: always_yes
DEBUG: conda config --set always_yes True
INFO: changed conda config: always_yes True
DEBUG: echo 'taxcalc . ' | tsort | xargs
INFO: saved previous working directory: /Users/mrh/work/OSPC/policybrain-builder
INFO: changed working directory: /tmp/policybrain-builder/pull/taxcalc
DEBUG: git rev-parse --is-inside-work-tree
INFO: restored previous working directory: /Users/mrh/work/OSPC/policybrain-builder
DEBUG: conda config --set always_yes False

INFO: restored previous conda config: always_yes
FINISHED : Mon Aug 28 19:06:42 EDT 2017
iMac2:policybrain-builder mrh$ 

[jbcrail]

I suspect the git command failed, but we currently don't print an error message in the event of an exception from a command.

Could you run these commands?

cd /tmp/policybrain-builder/pull/taxcalc
git rev-parse --is-inside-work-tree
echo $?

If this returns an error, then rm -rf /tmp/policybrain-builder should resolve it.

Either way, I'll add an error message during command execution to make sure users can spot an error sooner.

[martinholmer]

@jbcrail, Removing the /tmp/policybrain-builder/pull/taxcalc tree seems to help. Now my taxcalc-package-cli.sh script is working. I'll report on what happens with the 15 package builds and uploads.

Thanks for the fast help. Did I do something wrong?

Here is what happened when I followed your directions:

iMac2:~ mrh$ cd /tmp/policybrain-builder/pull/taxcalc
iMac2:taxcalc mrh$ pwd
/tmp/policybrain-builder/pull/taxcalc
iMac2:taxcalc mrh$ git rev-parse --is-inside-work-tree
fatal: Not a git repository (or any of the parent directories): .git
iMac2:taxcalc mrh$ echo $?
128
iMac2:taxcalc mrh$ cd ~
iMac2:.. mrh$ rm -rf /tmp/policybrain-builder
iMac2:.. mrh$ cd /tmp/policybrain-builder/pull/taxcalc
-bash: cd: /tmp/policybrain-builder/pull/taxcalc: No such file or directory
iMac2:.. mrh$ 

[martinholmer]

@jbcrail, The pb release ... command worked fine. What did I do wrong? I appreciate the great help from you, but I'd like to know how to fix things myself so I don't have to bother you.

[jbcrail]

@martinholmer, you didn't cause the error. I don't know the specific cause but it looks like the cached taxcalc repo left itself in a broken state somehow. It's hard to tell from the output.

There's likely an unhandled edge case. Since the CLI caches the cloned repos in the working directory between each run to help overall performance, it leaves us open to similar errors in the future. Removing the working directory between runs would eliminate this, but it would require a full clone each time (especially costly for B-Tax or OG-USA).

[martinholmer]

@jbcrail, I added rm -rf /tmp/policybrain-builder just before the pb release ... command in my taxcalc-package-cli.sh script. So, we don't slow down pb release, but still clean-up before starting. Hopefully this approach will avoid the problem I encountered in issue #61.

Thanks again for the pb CLI capability and the quick and effective help you've provided several times in the past few week. Your effort on this project is much appreciated.

@MattHJensen