pchanial / pyoperators

Operators and solvers for high-performance computing.
http://pchanial.github.com/pyoperators
Other
14 stars 9 forks source link

Add support for Python 3 #13

Closed ghisvail closed 9 years ago

ghisvail commented 10 years ago

Following what we discussed in #12, I think Python 3 support could be a great addition and should not be too difficult to achieve. I propose we log our progress here.

ghisvail commented 9 years ago

I have pushed a branch on my personal fork with an initial transformation to Python 3 via 2to3.

https://github.com/ghisvail/pyoperators/tree/enh/python3-support

The test suite does not run though.

ghisvail commented 9 years ago

FYI, I am trying to convert the whole codebase to Python 3 with 2to3 and then retrospectively introduce support for Python 2.7. Then you'll have a hybrid Python 2 / 3 codebase.

Another option is to have a Python 2 codebase and call 2to3 at install time, but that means you need to be sure that the result of 2to3 is spotless.

Regardless of the solution 2to3 is a good place to start with. Then, since I don't know all the specifics of the package and its test suite, your help would be more than welcome :)

ghisvail commented 9 years ago

More infos here: http://python3porting.com/strategies.html

pchanial commented 9 years ago

Thanks for all the pointers, I've started working on it. From now on, I'll write the codebase for Python 3, with a sprinkle here and there for Python 2 compatibility. Now I need to make the extensions work.

ghisvail commented 9 years ago

If you think you can make a new release compatible with Python 3 in time for the freeze, then I'll wait for it. Otherwise, I am happy to submit the latest 0.12.x for inclusion to the Debian archive.

pchanial commented 9 years ago

I think I've finished the conversion. Let me know if it's ok for you and I'll tag it as 0.12.15.

ghisvail commented 9 years ago

Ok I'll check out the latest commit, build and run the test suite for Python 3. If that passes, then I'll assume it's ready.

ghisvail commented 9 years ago

Hey Pierre,

Good news the project builds nicely. However the test suite is broken.

I fixed the test command which should be using nosetests3 instead in case Python 3 is used. I have submitted a PR for it, but some other commands may need a similar workaround too.

Then the test suite fails a bunch of tests. Here is a dump of my log:

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<class 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest self.test(self.arg) File "/home/gv10/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python3/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'DenseOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<class 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest self.test(self.arg) File "/home/gv10/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python3/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'HomothetyOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<class 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest self.test(self.arg) File "/home/gv10/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python3/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'SquareOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<class 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest self.test(self.arg) File "/home/gv10/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python3/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'Variable' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<class 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest self.test(_self.arg) File "/home/gv10/Workspace/pyoperators/test/test_core.py", line 355, in func assert_raises(TypeError, eval, 'x * y', {'x': x, 'y': y}) File "/home/gv10/Workspace/pyoperators/pyoperators/utils/testing.py", line 225, in assert_raises np.testing.assert_raises(_args, kwargs) File "/usr/lib/python3/dist-packages/numpy/testing/utils.py", line 1020, in assert_raises return nose.tools.assert_raises(*args,*kwargs) File "/usr/lib/python3.4/unittest/case.py", line 704, in assertRaises return context.handle('assertRaises', callableObj, args, kwargs) File "/usr/lib/python3.4/unittest/case.py", line 162, in handle callable_obj(_args, _kwargs) File "", line 1, in File "/usr/lib/python3/dist-packages/scipy/sparse/base.py", line 324, in mul** if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'VariableTranspose' object has no attribute 'ndim'

pchanial commented 9 years ago

which scipy version are you using? I'm running scipy 0.13.3 with python3 and I don't have these errors.

ghisvail commented 9 years ago

I am on 0.14.0, which is the latest version packaged on Debian Jessie / unstable.

ghisvail commented 9 years ago

I did not paste all the errors, but it seems they are all related to scipy indeed

ghisvail commented 9 years ago

Here is the full dump:

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'DenseOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'HomothetyOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'SquareOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'Variable' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 355, in func assert_raises(TypeError, eval, 'x * y', {'x': x, 'y': y}) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/utils/testing.py", line 225, in assert_raises np.testing.assert_raises(_args, kwargs) File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1020, in assert_raises return nose.tools.assert_raises(*args,*kwargs) File "/usr/lib/python2.7/unittest/case.py", line 473, in assertRaises callableObj(_args, _kwargs) File "", line 1, in File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul** if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'VariableTranspose' object has no attribute 'ndim'

ERROR: test.test_sparse.test_matvec('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assert_same(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_matvec('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assert_same(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_matvec('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assert_same(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_matvec('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assert_same(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_matvec('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assert_same(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_shapes('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assert_same(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_shapes('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assert_same(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_shapes('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assert_same(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_shapes('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assert_same(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_shapes('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assert_same(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_update_output('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_update_output('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_update_output('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_update_output('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'

ERROR: test.test_sparse.test_update_output('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools'


ghisvail commented 9 years ago

And I can do a from scipy.sparse import sparsetools or a import scipy.sparse.sparsetools without an AttributeError on a separate Python interpreter.

pchanial commented 9 years ago

OK, I can reproduce the errors using scipy 0.14...

2014-10-09 21:49 GMT+02:00 ghisvail notifications@github.com:

Here is the full dump: ERROR: test.test_core.test_times_mul_or_comp('

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testcore.py", line 359, in func z = x * y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in *mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'DenseOperator' object has no attribute 'ndim' ERROR: test.test_core.test_times_mul_or_comp('

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testcore.py", line 359, in func z = x * y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in *mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'HomothetyOperator' object has no attribute 'ndim' ERROR: test.test_core.test_times_mul_or_comp('

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testcore.py", line 359, in func z = x * y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in *mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'SquareOperator' object has no attribute 'ndim' ERROR: test.test_core.test_times_mul_or_comp('

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testcore.py", line 359, in func z = x * y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in *mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'Variable' object has no attribute 'ndim' ERROR: test.test_core.test_times_mul_or_comp('

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 355, in func assert_raises(TypeError, eval, 'x * y', {'x': x, 'y': y}) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/utils/testing.py", line 225, in assert_raises np.testing.assert_raises(_args, *kwargs) File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1020, in assert_raises return nose.tools.assert_raises(args,*kwargs) File "/usr/lib/python2.7/unittest/case.py", line 473, in assertRaises callableObj(_args, _kwargs) File "", line 1, in File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'VariableTranspose' object has no attribute 'ndim' ERROR: test.test_sparse.test_matvec('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assertsame(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_matvec('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assertsame(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_matvec('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assertsame(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_matvec('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assertsame(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_matvec('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 26, in func assertsame(so(vec), np.dot(A, vec)) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_shapes('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assertsame(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_shapes('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assertsame(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_shapes('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assertsame(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_shapes('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assertsame(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_shapes('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_sparse.py", line 41, in func assertsame(so(np.reshape(vec, shapein)), File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_update_output('bsr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testsparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_update_output('csr',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testsparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_update_output('csc',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testsparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_update_output('coo',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testsparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec') AttributeError: 'module' object has no attribute 'sparsetools' ERROR: test.test_sparse.test_update_output('dia',)

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/testsparse.py", line 56, in func so(vec, out, operation=operator.iadd) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/core.py", line 361, in *call self.direct(i, o, operation=operation) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/linear.py", line 426, in direct fn = getattr(sp.sparsetools, m.format + '_matvec')

AttributeError: 'module' object has no attribute 'sparsetools'

— Reply to this email directly or view it on GitHub https://github.com/pchanial/pyoperators/issues/13#issuecomment-58566720.

pchanial commented 9 years ago

So this one is an easy fix, thanks!

2014-10-09 21:51 GMT+02:00 ghisvail notifications@github.com:

And I can do a from scipy.sparse import sparsetools or a import scipy.sparse.sparsetools without an AttributeError on a separate Python interpreter.

— Reply to this email directly or view it on GitHub https://github.com/pchanial/pyoperators/issues/13#issuecomment-58566969.

ghisvail commented 9 years ago

Yes I manage to get around these errors by explictly importing import scipy.sparse.sparsetools as ssp

and replacing all occurences of sp.sparsetools by ssp in pyoperators/linear.py

The missing ndim attribute error remains though.

ghisvail commented 9 years ago

This is the dump of the last 5 errors:

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'DenseOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'HomothetyOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'SquareOperator' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 359, in func z = x \ y File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'Variable' object has no attribute 'ndim'

ERROR: test.test_core.test_times_mul_or_comp(<3x3 sparse matrix of type '<type 'numpy.int64'>'

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest self.test(_self.arg) File "/home/ghislain-deb/Workspace/pyoperators/test/test_core.py", line 355, in func assert_raises(TypeError, eval, 'x * y', {'x': x, 'y': y}) File "/home/ghislain-deb/Workspace/pyoperators/pyoperators/utils/testing.py", line 225, in assert_raises np.testing.assert_raises(_args, kwargs) File "/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1020, in assert_raises return nose.tools.assert_raises(*args,*kwargs) File "/usr/lib/python2.7/unittest/case.py", line 473, in assertRaises callableObj(_args, _kwargs) File "", line 1, in File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 324, in mul** if other.ndim == 1 or other.ndim == 2 and other.shape[1] == 1: AttributeError: 'VariableTranspose' object has no attribute 'ndim'


Ran 6399 tests in 22.025s

FAILED (SKIP=14, errors=5)

pchanial commented 9 years ago

So, I've fixed the test failures. For the ndim one, I was explicitly expecting 0.14 to fix a bug that they didn't quite fix. I should submit a more robust patch.

ghisvail commented 9 years ago

I confirm this, both Python 2 and 3 pass the test suite now. Good job Pierre.

pchanial commented 9 years ago

Thanks a lot for your help! Does the -mnose fix work for you, should I upload it?

ghisvail commented 9 years ago

-mnose fails as I mentioned in the PR thread.

It's not a terrible problem though, as long as the test suite can be run on one of the Python versions.

pchanial commented 9 years ago

I just tried the sys.executable -mnose fix and it works for me in both python 2 & 3. Are you sure that you built the extensions inplace (python3 setup.py build_ext --inplace)? I guess we should use the installed pyoperators package instead of the local one for test. Not sure what to do here.

ghisvail commented 9 years ago

I did yes:

python3 setup.py build_ext --inplace

followed by:

python3 setup.py test

and that generates a bunch of ImportError and NameError. No problem with Python 2 though. Could well be a bug in nose too.

If it definitely works on your Arch box, then you should probably release this version. I'll make sure the test suite is not run for Python 3 for now.

pchanial commented 9 years ago

Work is done, closing.