jeffgortmaker / pyblp

BLP Demand Estimation with Python
https://pyblp.readthedocs.io
MIT License
235 stars 81 forks source link

Trouble Getting Started #22

Closed shoshievass closed 5 years ago

shoshievass commented 5 years ago

I'm using python 3.6 through Anaconda. I installed pyblp through pip with no problems and tried running through the nested logit tutorial.

For some reason, the pyblp.Formulation command raised an assertion error. I'm sure the issue is something stupid, but I'm wondering if it might have to do with a dependency on patsy or something else? Any pointers you might have on how to resolve this?

image

jeffgortmaker commented 5 years ago

Can you post the full traceback so I can see what failed an assertion error?

If this is related to #19, it'd also help if you post the output of conda list. You might be on an old patsy version or something like that.

shoshievass commented 5 years ago

I'm updating conda now to make sure -- I'll post if the error doesn't go away.

shoshievass commented 5 years ago

It didn't work. Here is the full traceback:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-4-6733810cf6e0> in <module>()
----> 1 logit_formulation = pyblp.Formulation('prices', absorb='C(product_ids)')
      2 logit_formulation

~/anaconda3/lib/python3.6/site-packages/pyblp/configurations/formulation.py in __init__(self, formula, absorb, absorb_method)
    132         self._formula = formula
    133         self._absorb = absorb
--> 134         self._terms = parse_terms(formula)
    135         self._absorbed_terms: List[patsy.desc.Term] = []
    136         if absorb is not None:

~/anaconda3/lib/python3.6/site-packages/pyblp/configurations/formulation.py in parse_terms(formula)
    504 def parse_terms(formula: str) -> List[patsy.desc.Term]:
    505     """Parse patsy terms from a string. Validate that the string contains only right-hand side terms."""
--> 506     description = patsy.highlevel.ModelDesc.from_formula(formula)
    507     if description.lhs_termlist:
    508         end = formula.index('~') + 1 if '~' in formula else len(formula)

~/anaconda3/lib/python3.6/site-packages/patsy/desc.py in from_formula(cls, tree_or_string)
    162             tree = tree_or_string
    163         else:
--> 164             tree = parse_formula(tree_or_string)
    165         value = Evaluator().eval(tree, require_evalexpr=False)
    166         assert isinstance(value, cls)

~/anaconda3/lib/python3.6/site-packages/patsy/parse_formula.py in parse_formula(code, extra_operators)
    146     tree = infix_parse(_tokenize_formula(code, operator_strings),
    147                        operators,
--> 148                        _atomic_token_types)
    149     if not isinstance(tree, ParseNode) or tree.type != "~":
    150         tree = ParseNode("~", None, [tree], tree.origin)

~/anaconda3/lib/python3.6/site-packages/patsy/infix_parser.py in infix_parse(tokens, operators, atomic_types, trace)
    208 
    209     want_noun = True
--> 210     for token in token_source:
    211         if c.trace:
    212             print("Reading next token (want_noun=%r)" % (want_noun,))

~/anaconda3/lib/python3.6/site-packages/patsy/parse_formula.py in _tokenize_formula(code, operator_strings)
     92         else:
     93             it.push_back((pytype, token_string, origin))
---> 94             yield _read_python_expr(it, end_tokens)
     95 
     96 def test__tokenize_formula():

~/anaconda3/lib/python3.6/site-packages/patsy/parse_formula.py in _read_python_expr(it, end_tokens)
     42     origins = []
     43     bracket_level = 0
---> 44     for pytype, token_string, origin in it:
     45         assert bracket_level >= 0
     46         if bracket_level == 0 and token_string in end_tokens:

~/anaconda3/lib/python3.6/site-packages/patsy/util.py in next(self)
    330         else:
    331             # May raise StopIteration
--> 332             return six.advance_iterator(self._it)
    333     __next__ = next
    334 

~/anaconda3/lib/python3.6/site-packages/patsy/tokens.py in python_tokenize(code)
     33                 break
     34             origin = Origin(code, start, end)
---> 35             assert pytype not in (tokenize.NL, tokenize.NEWLINE)
     36             if pytype == tokenize.ERRORTOKEN:
     37                 raise PatsyError("error tokenizing input "

AssertionError: 
shoshievass commented 5 years ago

Conda list:

# packages in environment at /Users/shoshievass/anaconda3:
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py36h2fc01ae_0  
alabaster                 0.7.10           py36h174008c_0  
anaconda-client           1.6.14                   py36_0  
anaconda-navigator        1.9.6                    py36_0  
anaconda-project          0.8.2            py36h9ee5d53_0  
appnope                   0.1.0            py36hf537a9a_0  
appscript                 1.0.1            py36h9e71e49_1  
asn1crypto                0.24.0                   py36_0  
astroid                   1.6.3                    py36_0  
astropy                   3.0.2            py36h917ab60_1  
attrs                     18.1.0                   py36_0  
babel                     2.5.3                    py36_0  
backcall                  0.1.0                    py36_0  
backports                 1.0              py36ha3c1827_1  
backports.shutil_get_terminal_size 1.0.0            py36hd7a2ee4_2  
beautifulsoup4            4.6.0            py36h72d3c9f_1  
bitarray                  0.8.1            py36h1de35cc_1  
bkcharts                  0.2              py36h073222e_0  
blas                      1.0                         mkl  
blaze                     0.11.3           py36h02e7a37_0  
bleach                    2.1.3                    py36_0  
blosc                     1.14.3               hd9629dc_0  
bokeh                     0.12.16                  py36_0  
boto                      2.48.0           py36hdbc59ac_1  
bottleneck                1.2.1            py36hbd380ad_0  
bzip2                     1.0.6                h1de35cc_5  
ca-certificates           2019.1.23                     0  
certifi                   2018.11.29               py36_0  
cffi                      1.11.5           py36h342bebf_0  
chardet                   3.0.4            py36h96c241c_1  
click                     6.7              py36hec950be_0  
cloudpickle               0.5.3                    py36_0  
clyent                    1.2.2            py36hae3ad88_0  
colorama                  0.3.9            py36hd29a30c_0  
conda                     4.6.7                    py36_0  
conda-build               3.10.5                   py36_0  
conda-env                 2.6.0                h36134e3_0  
conda-verify              2.0.0            py36he837df3_0  
contextlib2               0.5.5            py36hd66e5e7_0  
cryptography              2.5              py36ha12b0ac_0  
curl                      7.63.0            ha441bb4_1000  
cycler                    0.10.0           py36hfc81398_0  
cython                    0.28.2           py36h1de35cc_0  
cytoolz                   0.9.0.1          py36h1de35cc_0  
dask                      0.17.5                   py36_0  
dask-core                 0.17.5                   py36_0  
datashape                 0.5.4            py36hfb22df8_0  
dbus                      1.13.2               h760590f_1  
decorator                 4.3.0                    py36_0  
distributed               1.21.8                   py36_0  
docutils                  0.14             py36hbfde631_0  
entrypoints               0.2.3            py36hd81d71f_2  
et_xmlfile                1.0.1            py36h1315bdc_0  
expat                     2.2.5                hb8e80ba_0  
fastcache                 1.0.2            py36h1de35cc_2  
filelock                  3.0.4                    py36_0  
flask                     1.0.2                    py36_1  
flask-cors                3.0.4                    py36_0  
freetype                  2.8                  h12048fb_1  
future                    0.16.0                   pypi_0    pypi
get_terminal_size         1.0.0                h7520d66_0  
gettext                   0.19.8.1             h15daf44_3  
gevent                    1.3.0            py36h1de35cc_0  
glib                      2.56.1               h35bc53a_0  
glob2                     0.6              py36h94c9186_0  
gmp                       6.1.2                hb37e062_1  
gmpy2                     2.0.8            py36hf9c35bd_2  
greenlet                  0.4.13           py36h1de35cc_0  
h5py                      2.7.1            py36ha8ecd60_2  
hdf5                      1.10.2               hfa1e0ec_1  
heapdict                  1.0.0                    py36_2  
html5lib                  1.0.1            py36h2f9c1c0_0  
icu                       58.2                 h4b95b61_1  
idna                      2.6              py36h8628d0a_1  
imageio                   2.3.0                    py36_0  
imagesize                 1.0.0                    py36_0  
intel-openmp              2018.0.0                      8  
ipykernel                 4.8.2                    py36_0  
ipython                   6.4.0                    py36_0  
ipython_genutils          0.2.0            py36h241746c_0  
ipywidgets                7.2.1                    py36_0  
isort                     4.3.4                    py36_0  
itsdangerous              0.24             py36h49fbb8d_1  
jbig                      2.1                  h4d881f8_0  
jdcal                     1.4                      py36_0  
jedi                      0.12.0                   py36_1  
jinja2                    2.10             py36hd36f9c5_0  
jpeg                      9b                   he5867d9_2  
jsonschema                2.6.0            py36hb385e00_0  
jupyter                   1.0.0                    py36_4  
jupyter_client            5.2.3                    py36_0  
jupyter_console           5.2.0            py36hccf5b1c_1  
jupyter_core              4.4.0            py36h79cf704_0  
jupyterlab                0.32.1                   py36_0  
jupyterlab_launcher       0.10.5                   py36_0  
keyring                   18.0.0                   py36_0  
kiwisolver                1.0.1            py36h792292d_0  
krb5                      1.16.1               hddcf347_7  
lazy-object-proxy         1.3.1            py36h2fbbe47_0  
libcurl                   7.63.0            h051b688_1000  
libcxx                    4.0.1                h579ed51_0  
libcxxabi                 4.0.1                hebd6815_0  
libedit                   3.1.20170329         hb402a30_2  
libffi                    3.2.1                h475c297_4  
libgfortran               3.0.1                h93005f0_2  
libiconv                  1.15                 hdd342a3_7  
libpng                    1.6.34               he12f830_0  
libsodium                 1.0.16               h3efe00b_0  
libssh2                   1.8.0                ha12b0ac_4  
libtiff                   4.0.9                hcb84e12_1  
libxml2                   2.9.8                hab757c2_1  
libxslt                   1.1.32               hb819dd2_0  
llvmlite                  0.23.1           py36hc454e04_0  
locket                    0.2.0            py36hca03003_1  
lxml                      4.2.1            py36h7166777_0  
lzo                       2.10                 h362108e_2  
markupsafe                1.0              py36h3a1e703_1  
matplotlib                2.2.2            py36ha7267d0_0  
mccabe                    0.6.1            py36hdaeb55d_0  
mistune                   0.8.3            py36h1de35cc_1  
mkl                       2018.0.2                      1  
mkl-service               1.1.2            py36h7ea6df4_4  
mkl_fft                   1.0.1            py36h917ab60_0  
mkl_random                1.0.1            py36h78cc56f_0  
more-itertools            4.1.0                    py36_0  
mpc                       1.0.3                h7a72875_5  
mpfr                      3.1.5                h711e7fd_2  
mpmath                    1.0.0            py36hf1b8295_2  
msgpack                   0.5.6                    pypi_0    pypi
msgpack-python            0.5.6            py36h04f5b5a_0  
multipledispatch          0.5.0                    py36_0  
navigator-updater         0.2.1                    py36_0  
nbconvert                 5.3.1            py36h810822e_0  
nbformat                  4.4.0            py36h827af21_0  
ncurses                   6.1                  h0a44026_0  
networkx                  2.1                      py36_0  
nltk                      3.3.0                    py36_0  
nose                      1.3.7            py36h73fae2b_2  
notebook                  5.5.0                    py36_0  
numba                     0.38.0           py36h1702cab_0  
numexpr                   2.6.5            py36h057f876_0  
numpy                     1.14.3           py36h9bb19eb_1  
numpy-base                1.14.3           py36h479e554_1  
numpydoc                  0.8.0                    py36_0  
odo                       0.5.1            py36hc1af34a_0  
olefile                   0.45.1                   py36_0  
openpyxl                  2.5.3                    py36_0  
openssl                   1.1.1b               h1de35cc_0  
packaging                 17.1                     py36_0  
pandas                    0.23.0           py36h1702cab_0  
pandoc                    1.19.2.1             ha5e8f32_1  
pandocfilters             1.4.2            py36h3b0b094_1  
parso                     0.2.0                    py36_0  
partd                     0.3.8            py36hf5c4cb8_0  
path.py                   11.0.1                   py36_0  
pathlib2                  2.3.2                    py36_0  
patsy                     0.5.0                    py36_0  
pcre                      8.42                 h378b8a2_0  
pep8                      1.7.1                    py36_0  
pexpect                   4.5.0                    py36_0  
pickleshare               0.7.4            py36hf512f8e_0  
pillow                    5.1.0            py36hfcce615_0  
pip                       19.0.3                   pypi_0    pypi
pkginfo                   1.4.2                    py36_1  
pluggy                    0.6.0            py36hb1d0581_0  
ply                       3.11                     py36_0  
prompt_toolkit            1.0.15           py36haeda067_0  
proselint                 0.10.2                   pypi_0    pypi
psutil                    5.4.5            py36h1de35cc_0  
ptyprocess                0.5.2            py36he6521c3_0  
py                        1.5.3                    py36_0  
pyblp                     0.6.1                    pypi_0    pypi
pycodestyle               2.4.0                    py36_0  
pycosat                   0.6.3            py36hee92d8f_0  
pycparser                 2.18             py36h724b2fc_1  
pycrypto                  2.6.1            py36h1de35cc_8  
pycurl                    7.43.0.2         py36ha12b0ac_0  
pyflakes                  1.6.0            py36hea45e83_0  
pygments                  2.2.0            py36h240cd3f_0  
pylint                    1.8.4                    py36_0  
pyodbc                    4.0.23           py36h0a44026_0  
pyopenssl                 18.0.0                   py36_0  
pyparsing                 2.2.0            py36hb281f35_0  
pyqt                      5.9.2            py36h11d3b92_0  
pysocks                   1.6.8                    py36_0  
pytables                  3.4.3            py36h5ca999c_2  
pytest                    3.5.1                    py36_0  
pytest-arraydiff          0.2                      py36_0  
pytest-astropy            0.3.0                    py36_0  
pytest-doctestplus        0.1.3                    py36_0  
pytest-openfiles          0.3.0                    py36_0  
pytest-remotedata         0.2.1                    py36_0  
python                    3.6.8                haf84260_0  
python-dateutil           2.7.3                    py36_0  
python.app                2                        py36_8  
pytz                      2018.4                   py36_0  
pywavelets                0.5.2            py36h2710a04_0  
pyyaml                    3.12             py36h2ba1e63_1  
pyzmq                     17.0.0           py36h1de35cc_1  
qt                        5.9.5                h02808f3_0  
qtawesome                 0.4.4            py36h468c6fb_0  
qtconsole                 4.3.1            py36hd96c0ff_0  
qtpy                      1.4.1                    py36_0  
readline                  7.0                  hc1231fa_4  
requests                  2.18.4           py36h4516966_1  
rope                      0.10.7           py36h68959ac_0  
ruamel_yaml               0.15.35          py36h1de35cc_1  
scikit-image              0.13.1           py36h1de35cc_1  
scikit-learn              0.19.1           py36hffbff8c_0  
scipy                     1.1.0            py36hcaad992_0  
seaborn                   0.8.1            py36h595ecd9_0  
send2trash                1.5.0                    py36_0  
setuptools                39.1.0                   py36_0  
simplegeneric             0.8.1                    py36_2  
singledispatch            3.4.0.3          py36hf20db9d_0  
sip                       4.19.8           py36h0a44026_0  
six                       1.11.0           py36h0e22d5e_1  
snappy                    1.1.7                he62c110_3  
snowballstemmer           1.2.1            py36h6c7b616_0  
sortedcollections         0.6.1                    py36_0  
sortedcontainers          1.5.10                   py36_0  
sphinx                    1.7.4                    py36_0  
sphinxcontrib             1.0              py36h9364dc8_1  
sphinxcontrib-websupport  1.0.1            py36h92f4a7a_1  
spyder                    3.3.1                    py36_1  
spyder-kernels            0.2.6                    py36_0  
sqlalchemy                1.2.7            py36hb402a30_0  
sqlite                    3.26.0               ha441bb4_0  
statsmodels               0.9.0            py36h917ab60_0  
sympy                     1.1.1            py36h7f3cf04_0  
tblib                     1.3.2            py36hda67792_0  
terminado                 0.8.1                    py36_1  
testpath                  0.3.1            py36h625a49b_0  
tk                        8.6.8                ha441bb4_0  
toolz                     0.9.0                    py36_0  
tornado                   5.0.2                    py36_0  
traitlets                 4.3.2            py36h65bd3ce_0  
typing                    3.6.4                    py36_0  
unicodecsv                0.14.1           py36he531d66_0  
unixodbc                  2.3.6                h3efe00b_0  
urllib3                   1.22             py36h68b9469_0  
wcwidth                   0.1.7            py36h8c6ec74_0  
webencodings              0.5.1            py36h3b9701d_1  
werkzeug                  0.14.1                   py36_0  
wheel                     0.31.1                   py36_0  
widgetsnbextension        3.2.1                    py36_0  
wrapt                     1.10.11          py36hc29e774_0  
xlrd                      1.1.0            py36h336f4a2_1  
xlsxwriter                1.0.4                    py36_0  
xlwings                   0.11.8                   py36_0  
xlwt                      1.2.0            py36h5ad1178_0  
xz                        5.2.4                h1de35cc_4  
yaml                      0.1.7                hc338f04_2  
zeromq                    4.2.5                h378b8a2_0  
zict                      0.1.3            py36h71da714_0  
zlib                      1.2.11               hf3cbc9b_2  
jeffgortmaker commented 5 years ago

Looks like you still have an outdated patsy version (0.5.0 instead of 0.5.1).

Can you try conda update patsy?

shoshievass commented 5 years ago

Solved it! Thank you :)

jeffgortmaker commented 5 years ago

Glad it worked! I'll leave this issue open for now as a reminder to add some docs about outdated requirements.