search

PSLmodels / Package-Builder

Policy Simulation Library tool that builds packages and uploads them to the Anaconda Cloud's PSLmodels channel for public distribution
4 stars 7 forks source link

"dirty" is in version name if using `builder.py` #88

Closed hdoupe closed 5 years ago

hdoupe commented 5 years ago

In https://github.com/open-source-economics/OG-USA/issues/394, @martinholmer pointed out that "dirty" is in the OG-USA package version. This usually happens if you are working with a package locally and have made changes that have not been committed yet:

If either is called in a checked out source tree, they invoke “git describe” (with –tags –always –dirty) to come up with a fine-grained version string. If you’re sitting on a tag, you get just “1.4”. If you’re after a tag, you’ll get something like “1.4-8-gf7283c2”, which means there are 8 commits after the 1.4 tag, and the abbreviated SHA1 revision ID is f7283c2. And if your tree has uncommitted changes, you’ll get “1.4-8-gf7283c2-dirty”.

source: https://blog.mozilla.org/warner/2012/01/31/version-string-management-in-python-introducing-python-versioneer/

For example:

HDoupe-MacBook-Pro:Documents henrydoupe$ mkdir dirtytest
HDoupe-MacBook-Pro:Documents henrydoupe$ cd dirtytest/
HDoupe-MacBook-Pro:dirtytest henrydoupe$ git clone https://github.com/open-source-economics/Tax-Calculator
Cloning into 'Tax-Calculator'...
remote: Enumerating objects: 92, done.
remote: Counting objects: 100% (92/92), done.
remote: Compressing objects: 100% (60/60), done.
remote: Total 29332 (delta 40), reused 58 (delta 32), pack-reused 29240
Receiving objects: 100% (29332/29332), 982.97 MiB | 6.18 MiB/s, done.
Resolving deltas: 100% (21824/21824), done.
HDoupe-MacBook-Pro:dirtytest henrydoupe$ cd Tax-Calculator/
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ git checkout 0.20.1 -b v0.20.1
Switched to a new branch 'v0.20.1'
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ echo "import taxcalc; print(taxcalc.__version__)" > taxcalc/extra.py
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ pip install -e .
Obtaining file:///Users/henrydoupe/Documents/dirtytest/Tax-Calculator
Requirement already satisfied: numpy in /Users/henrydoupe/anaconda3/lib/python3.6/site-packages (from taxcalc==0.20.1) (1.15.1)
Requirement already satisfied: pandas in /Users/henrydoupe/anaconda3/lib/python3.6/site-packages (from taxcalc==0.20.1) (0.23.4)
Requirement already satisfied: python-dateutil>=2.5.0 in /Users/henrydoupe/.local/lib/python3.6/site-packages (from pandas->taxcalc==0.20.1) (2.6.1)
Requirement already satisfied: pytz>=2011k in /Users/henrydoupe/anaconda3/lib/python3.6/site-packages (from pandas->taxcalc==0.20.1) (2018.5)
Requirement already satisfied: six>=1.5 in /Users/henrydoupe/.local/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas->taxcalc==0.20.1) (1.11.0)
pyasn1-modules 0.2.2 has requirement pyasn1<0.5.0,>=0.4.1, but you'll have pyasn1 0.3.7 which is incompatible.
Installing collected packages: taxcalc
  Found existing installation: taxcalc 0.8.4-2078-gc8014470
    Uninstalling taxcalc-0.8.4-2078-gc8014470:
      Successfully uninstalled taxcalc-0.8.4-2078-gc8014470
  Running setup.py develop for taxcalc
Successfully installed taxcalc
You are using pip version 18.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ python -c "import taxcalc; print(taxcalc.__version__)"
0.20.1
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ git status
On branch v0.20.1
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    taxcalc/extra.py

nothing added to commit but untracked files present (use "git add" to track)
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ git add taxcalc/extra.py 
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ python taxcalc/extra.py 
0.20.1-dirty
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ git commit -m "extra file"
[v0.20.1 8d0b2946] extra file
 1 file changed, 1 insertion(+)
 create mode 100644 taxcalc/extra.py
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$ python taxcalc/extra.py 
0.20.1-1-g8d0b2946
HDoupe-MacBook-Pro:Tax-Calculator henrydoupe$

My hunch is that this is related to writing over the version set in the replace_version function. Basically, what it's making an edit to a file and saving it. I think this is making the version state "dirty" in the way described above.

martinholmer commented 5 years ago

@hdoupe said:

My hunch is that this is related to writing over the version set in the replace_version function [in builder.py]. Basically, what it's making an edit to a file and saving it. I think this is making the version state "dirty" in the way described above.

So, it sounds as if the builder.py script could use some more work.

hdoupe commented 5 years ago

Right, the scripts in the policybrain_builder/policybrain_builder directory don't have this problem, and they do a similar thing to fix the version. So, the right place to start would probably be trying to figure out how they solve this issue.

I'm going out of town for a week. If anyone else wants to take a crack at this problem, I'm happy to give feedback when I get back. Otherwise, I can take a look at it when I get back.

martinholmer commented 5 years ago

@hdoupe said in policybrain-builder issue #88:

Right, the scripts in the policybrain_builder/policybrain_builder directory don't have this problem, and they do a similar thing to fix the version. So, the right place to start would probably be trying to figure out how they solve this issue.

I'm going out of town for a week. If anyone else wants to take a crack at this problem, I'm happy to give feedback when I get back. Otherwise, I can take a look at it when I get back.

Frankly, I think it is an enormous waste of time to put any more effort into the builder.py script when we have a capability what works just fine. There are many more pressing problems with TaxBrain, Tax-Calculator tbi (PR 2043 has been open since August 6th), and PSL. Why would it make sense to divert staff time away from these more pressing issues when we already have a perfectly good capability in the policybrain_builder repo?

@MattHJensen

hdoupe commented 5 years ago

@martinholmer thanks for improving the policybrain-builder tool over the last few days. I added the builder.py script because I was unable to successfully use the policybrain-builder tool. I'm fine with removing this file in light of your improvements.

martinholmer commented 5 years ago

@hdoupe said in package-builder issue #88:

Thanks for improving the policybrain-builder tool over the last few days. I added the builder.py script because I was unable to successfully use the policybrain-builder tool. I'm fine with removing this file in light of your improvements.

OK. But let's wait a bit until I get my confidence up about the recent improvements. As described here, I'm in the midst of testing the B-Tax 0.2.1 packages I built and uploaded yesterday. And after that I want to see if the latest version of the package-builder can build and upload packages for a recent OG-USA release.

@jdebacker @rickecon @MattHJensen

martinholmer commented 5 years ago

The new version of Package-Builder resolves the versioning problem discussed in issue #88, and so, this issue is being closed.