pydata / patsy

Describing statistical models in Python using symbolic formulas
Other
956 stars 104 forks source link

statsmodels.formula.api tokenize issue in Python 3.7.1 #135

Closed sbwiecko closed 6 years ago

sbwiecko commented 6 years ago

https://github.com/statsmodels/statsmodels/issues/5343#issue-373873989

Hoeze commented 6 years ago
formula
Out[4]: '~ 1'
dmat = patsy.highlevel.dmatrix(formula, sample_description)
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2961, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-9ac6795558a7>", line 1, in <module>
    dmat = patsy.highlevel.dmatrix(formula, sample_description)
  File "/usr/lib/python3.7/site-packages/patsy/highlevel.py", line 291, in dmatrix
    NA_action, return_type)
  File "/usr/lib/python3.7/site-packages/patsy/highlevel.py", line 165, in _do_highlevel_design
    NA_action)
  File "/usr/lib/python3.7/site-packages/patsy/highlevel.py", line 62, in _try_incr_builders
    formula_like = ModelDesc.from_formula(formula_like)
  File "/usr/lib/python3.7/site-packages/patsy/desc.py", line 164, in from_formula
    tree = parse_formula(tree_or_string)
  File "/usr/lib/python3.7/site-packages/patsy/parse_formula.py", line 148, in parse_formula
    _atomic_token_types)
  File "/usr/lib/python3.7/site-packages/patsy/infix_parser.py", line 210, in infix_parse
    for token in token_source:
  File "/usr/lib/python3.7/site-packages/patsy/parse_formula.py", line 94, in _tokenize_formula
    yield _read_python_expr(it, end_tokens)
  File "/usr/lib/python3.7/site-packages/patsy/parse_formula.py", line 44, in _read_python_expr
    for pytype, token_string, origin in it:
  File "/usr/lib/python3.7/site-packages/patsy/util.py", line 332, in next
    return six.advance_iterator(self._it)
  File "/usr/lib/python3.7/site-packages/patsy/tokens.py", line 35, in python_tokenize
    assert pytype not in (tokenize.NL, tokenize.NEWLINE)
AssertionError
njsmith commented 6 years ago

This was fixed by #131; just need to do a release, which I'm working on now.