NumExpr 2.2.1 should be compatible with PyTables <= 2.4

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Running nosetests pandas/io/tests/test_pytables.py from a fresh pandas clone 
(after running python setup.py develop)

What is the expected output? What do you see instead?

I expect all tests to pass or be skipped.

What version of the product are you using? On what operating system?

NumExpr 2.2.1, PyTables 2.4, Arch Linux (doesn't matter, this also fails on 
Windows)

Please provide any additional information below.

Please add the copy_args kwarg back in or just remove the check or (preferably) 
just specify the allowable keyword arguments in the function signature.

If you're going to allow **kwargs then you shouldn't validate them. If you are 
going to validate them you should just explicitly use the kwargs you want in 
the function signature. FWIW **kwargs is a nightmare for backwards 
compatibility.

NumExpr 2.2.1 should maintain backwards compatibility with PyTables < 3.0.0

Original issue reported on code.google.com by cpcl...@gmail.com on 20 Sep 2013 at 8:45

[GoogleCodeExporter]

Here's a small example without needing pandas

import tables
import numpy as np
import datetime, time

test_file = 'test_select.h5'
handle = tables.openFile(test_file, "w")
node = handle.createGroup(handle.root, 'test')
table = handle.createTable(node, 'table',
                           dict(index=tables.Int64Col(),
                                column=tables.StringCol(25),
                                values=tables.FloatCol(shape=3)))

r = table.row
for i in range(10):
    r['index'] = i
    r['column'] = "str-%d" % (i % 5)
    r['values'] = np.arange(3)
    r.append()
table.flush()
selector = "(column == 'str-2')"
result = handle.root.test.table.readWhere(selector)
handle.close()

Original comment by cpcl...@gmail.com on 20 Sep 2013 at 9:33

[GoogleCodeExporter]

This change was made back in numexpr 2.2, which explicitly dropped support of 
PyTables < 3.0, as can be seen in the release notes:

https://code.google.com/p/numexpr/wiki/ReleaseNotes

If you need compatibility with PyTables < 3.0 please use numexpr < 2.2.

Sorry for the inconveniences.

Original comment by fal...@gmail.com on 21 Sep 2013 at 8:37

• Changed state: WontFix

[GoogleCodeExporter]

Can you please make it more clear? The line

[API CLEAN] Removed copy_args argument of evaluate. This should only be used by 
old versions of PyTables (< 3.0).

is not explicit at all. All it states is that there was (a completely 
unnecessary IMHO) breaking API change.

An explicit version might have read:

[COMPAT]: Dropped support for PyTables < 3.0.0

What's the reason for not keeping backward compatibility, especially if the 
only change is a single keyword argument? Are there other backwards 
incompatible changes?

Original comment by cpcl...@gmail.com on 21 Sep 2013 at 12:00

[GoogleCodeExporter]

We opened (and closed) an issue over at PyTables:

https://github.com/PyTables/PyTables/issues/283

we (pandas) would be eternally grateful if you would make numexpr 2.2 backwards 
compatible with PyTables < 3.0.0. The change is essentially a one line revert 
since the keyword argument copy_args has been doing nothing since 2.0.

Original comment by cpcl...@gmail.com on 21 Sep 2013 at 3:58

[GoogleCodeExporter]

Ok.  I have read the extensive comments in PyTables #283 and you convinced me 
that the best solution is to revert the old behaviour of `copy_args`.  Will do 
a new release as soon as I have some free time.

Original comment by fal...@gmail.com on 23 Sep 2013 at 8:34

• Changed state: Accepted

[GoogleCodeExporter]

I have tried to restore the copy_args attribute, but PyTables 2.4.0 still fails 
in other tests.  For example:

ERROR: test (tables.tests.test_queries.IndexedTableUsage25)

Traceback (most recent call last): File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/tests/common.py", line 259, in newmethod return oldmethod(self, _args, *_kwargs) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/tests/test_queries.py", line 705, in test c_usable_idxs = self.willQueryUseIndexing(condition, {}) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/table.py", line 1310, in willQueryUseIndexing compiled = self._compileCondition(condition, condvars) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/table.py", line 1284, in _compileCondition compiled = compile_condition(condition, typemap, indexedcols, copycols) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/conditions.py", line 377, in compile_condition idxexprs = _get_idx_expr(expr, indexedcols) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/conditions.py", line 274, in _get_idx_expr return _get_idx_expr_recurse(expr, indexedcols, [], ['']) File "/home/faltet/anaconda/lib/python2.7/site-packages/tables/conditions.py", line 205, in _get_idx_expr_recurse if lcolvar and rcolvar and lcolvar == rcolvar and op == 'and': File "/home/faltet/anaconda/lib/python2.7/site-packages/numexpr-2.2.2.dev-py2.7-linux-x86_64.egg/numexpr/expressions.py", line 400, in nonzero raise TypeError("You can't use Python's standard boolean operators in " TypeError: You can't use Python's standard boolean operators in NumExpr expressions. You should use their bitwise counterparts instead: '&' instead of 'and', '|' instead of 'or', and '~' instead of 'not'.

Are you seeing the same, or it is only my box?

Original comment by fal...@gmail.com on 25 Sep 2013 at 4:54

[GoogleCodeExporter]

Okay I have put the copy_args argument back and and commented again the 
`__nozero__` operator.  With this, branch 2.2 is compatible again with PyTables 
< 3.  However, default branch (probably to be released as 3.0) continues to be 
incompatible.

Original comment by fal...@gmail.com on 27 Sep 2013 at 8:13

• Changed state: Fixed

[GoogleCodeExporter]

Thanks for the really quick response!

Original comment by jeffreb...@gmail.com on 27 Sep 2013 at 12:25

[GoogleCodeExporter]

We really appreciate this! Thanks.

Original comment by cpcl...@gmail.com on 27 Sep 2013 at 12:31