search

Blosc / bcolz

A columnar data container that can be compressed.
http://bcolz.blosc.org
959 stars 149 forks source link

Something wrong with cython version detection #42

Closed esc closed 10 years ago

esc commented 10 years ago 
[ec2-user@ip-172-31-42-166 ~]$ cython --version    
Cython version 0.21
[ec2-user@ip-172-31-42-166 ~]$ pip install bcolz   
Downloading/unpacking bcolz
  Downloading bcolz-0.7.1.tar.gz (666kB): 666kB downloaded
  Running setup.py (path:/tmp/pip_build_ec2-user/bcolz/setup.py) egg_info for package bcolz
    .. ERROR:: You need Cython 0.20 or greater to compile bcolz!
    Complete output from command python setup.py egg_info:
    .. ERROR:: You need Cython 0.20 or greater to compile bcolz!

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_ec2-user/bcolz
Storing debug log for failure in /home/ec2-user/.pip/pip.log
esc commented 10 years ago

In cython 0.21 the following fails:

In [1]: from Cython.Compiler.Main import Version
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-ad365ee0a5dd> in <module>()
----> 1 from Cython.Compiler.Main import Version

ImportError: cannot import name Version
esc commented 10 years ago

Actually it seems like bcolz will not compile at all with Cython 0.21...

grenoya commented 10 years ago

The probelm is located in line 86 of the setup.py file. It should be: from Cython.Compiler.Main import version (please note the minus 'v' of 'version')

grenoya commented 10 years ago

or even better : it should use Cython. version

esc commented 10 years ago

So the fix seems to work OK, however then I get some issues cythoning:

* Found Cython 0.21 package installed.
* Found numpy 1.9.0 package installed.
* Found numexpr 2.3.1 package installed.
running install
running build
running build_py
copying bcolz/version.py -> build/lib.linux-x86_64-2.7/bcolz
running build_ext
cythoning bcolz/carray_ext.pyx to bcolz/carray_ext.c

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'definitions.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'import_array.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
                        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:25: Name 'import_array' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'ndarray.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
                                      ^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:39: Name 'ndarray' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'dtype.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
                                               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:48: Name 'dtype' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'malloc.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:4: Name 'malloc' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'realloc.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
           ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:12: Name 'realloc' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'free.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
                    ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:21: Name 'free' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'memcpy.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
                          ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:27: Name 'memcpy' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'memset.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
                                  ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:35: Name 'memset' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'strdup.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
                                          ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:43: Name 'strdup' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'strcmp.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
                                                  ^
------------------------------------------------------------

bcolz/carray_ext.pyx:63:51: Name 'strcmp' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyString_AsString.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:64:4: Name 'PyString_AsString' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyString_GET_SIZE.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
                      ^
------------------------------------------------------------

bcolz/carray_ext.pyx:64:23: Name 'PyString_GET_SIZE' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyString_FromStringAndSize.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:65:4: Name 'PyString_FromStringAndSize' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'Py_BEGIN_ALLOW_THREADS.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:66:4: Name 'Py_BEGIN_ALLOW_THREADS' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'Py_END_ALLOW_THREADS.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
                           ^
------------------------------------------------------------

bcolz/carray_ext.pyx:66:28: Name 'Py_END_ALLOW_THREADS' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyArray_GETITEM.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
    PyArray_GETITEM, PyArray_SETITEM, \
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:67:4: Name 'PyArray_GETITEM' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyArray_SETITEM.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
from definitions cimport import_array, ndarray, dtype, \
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
    PyArray_GETITEM, PyArray_SETITEM, \
                    ^
------------------------------------------------------------

bcolz/carray_ext.pyx:67:21: Name 'PyArray_SETITEM' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'npy_intp.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
    PyArray_GETITEM, PyArray_SETITEM, \
    npy_intp, PyBuffer_FromMemory, Py_uintptr_t
   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:68:4: Name 'npy_intp' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'PyBuffer_FromMemory.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
    PyArray_GETITEM, PyArray_SETITEM, \
    npy_intp, PyBuffer_FromMemory, Py_uintptr_t
             ^
------------------------------------------------------------

bcolz/carray_ext.pyx:68:14: Name 'PyBuffer_FromMemory' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
IntType = np.dtype(np.int_)

#-----------------------------------------------------------------

# numpy functions & objects
from definitions cimport import_array, ndarray, dtype, \
^
------------------------------------------------------------

bcolz/carray_ext.pyx:62:0: 'Py_uintptr_t.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
    malloc, realloc, free, memcpy, memset, strdup, strcmp, \
    PyString_AsString, PyString_GET_SIZE, \
    PyString_FromStringAndSize, \
    Py_BEGIN_ALLOW_THREADS, Py_END_ALLOW_THREADS, \
    PyArray_GETITEM, PyArray_SETITEM, \
    npy_intp, PyBuffer_FromMemory, Py_uintptr_t
                                  ^
------------------------------------------------------------

bcolz/carray_ext.pyx:68:35: Name 'Py_uintptr_t' not declared in module 'definitions'

Error compiling Cython file:
------------------------------------------------------------
...
        return [list_bytes_to_str(x) for x in lst]
    else:
        return lst

# This is the same than in utils.py, but works faster in extensions
cdef get_len_of_range(npy_intp start, npy_intp stop, npy_intp step):
                     ^
------------------------------------------------------------

bcolz/carray_ext.pyx:201:22: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        return [list_bytes_to_str(x) for x in lst]
    else:
        return lst

# This is the same than in utils.py, but works faster in extensions
cdef get_len_of_range(npy_intp start, npy_intp stop, npy_intp step):
                                     ^
------------------------------------------------------------

bcolz/carray_ext.pyx:201:38: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        return [list_bytes_to_str(x) for x in lst]
    else:
        return lst

# This is the same than in utils.py, but works faster in extensions
cdef get_len_of_range(npy_intp start, npy_intp stop, npy_intp step):
                                                    ^
------------------------------------------------------------

bcolz/carray_ext.pyx:201:53: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    if start < stop:
        # Do not use a cython.cdiv here (do not ask me why!)
        n = ((stop - start - 1) // step + 1)
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:211:16: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    if start < stop:
        # Do not use a cython.cdiv here (do not ask me why!)
        n = ((stop - start - 1) // step + 1)
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
                                ^
------------------------------------------------------------

bcolz/carray_ext.pyx:211:33: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        # Do not use a cython.cdiv here (do not ask me why!)
        n = ((stop - start - 1) // step + 1)
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
                npy_intp start, npy_intp stop, npy_intp step):
               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:212:16: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        # Do not use a cython.cdiv here (do not ask me why!)
        n = ((stop - start - 1) // step + 1)
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
                npy_intp start, npy_intp stop, npy_intp step):
                               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:212:32: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        # Do not use a cython.cdiv here (do not ask me why!)
        n = ((stop - start - 1) // step + 1)
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
                npy_intp start, npy_intp stop, npy_intp step):
                                              ^
------------------------------------------------------------

bcolz/carray_ext.pyx:212:47: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef object atom, constant, dobject

    cdef void _getitem(self, int start, int stop, char *dest)
    cdef compress_data(self, char *data, size_t itemsize, size_t nbytes,
                       object cparams)
    cdef compress_arrdata(self, ndarray array, int itemsize,
                               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:295:32: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        self.nbytes = nbytes
        self.cbytes = cbytes + footprint
        self.cdbytes = cbytes
        self.blocksize = blocksize

    cdef compress_arrdata(self, ndarray array, int itemsize,
                               ^
------------------------------------------------------------

bcolz/carray_ext.pyx:355:32: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
cdef class chunks(object):
    """Store the different carray chunks in a directory on-disk."""
    cdef object _rootdir, _mode
    cdef object dtype, cparams, lastchunkarr
    cdef object chunk_cached
    cdef npy_intp nchunks, nchunk_cached, len
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:658:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

    cdef public int itemsize, atomsize
    cdef int _chunksize, _chunklen, leftover
    cdef int nrowsinbuf, _row
    cdef int sss_mode, wheretrue_mode, where_mode
    cdef npy_intp startb, stopb
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:865:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef public int itemsize, atomsize
    cdef int _chunksize, _chunklen, leftover
    cdef int nrowsinbuf, _row
    cdef int sss_mode, wheretrue_mode, where_mode
    cdef npy_intp startb, stopb
    cdef npy_intp start, stop, step, nextelement
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:866:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef int _chunksize, _chunklen, leftover
    cdef int nrowsinbuf, _row
    cdef int sss_mode, wheretrue_mode, where_mode
    cdef npy_intp startb, stopb
    cdef npy_intp start, stop, step, nextelement
    cdef npy_intp _nrow, nrowsread
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:867:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef int nrowsinbuf, _row
    cdef int sss_mode, wheretrue_mode, where_mode
    cdef npy_intp startb, stopb
    cdef npy_intp start, stop, step, nextelement
    cdef npy_intp _nrow, nrowsread
    cdef npy_intp _nbytes, _cbytes
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:868:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef int sss_mode, wheretrue_mode, where_mode
    cdef npy_intp startb, stopb
    cdef npy_intp start, stop, step, nextelement
    cdef npy_intp _nrow, nrowsread
    cdef npy_intp _nbytes, _cbytes
    cdef npy_intp nhits, limit, skip
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:869:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef npy_intp startb, stopb
    cdef npy_intp start, stop, step, nextelement
    cdef npy_intp _nrow, nrowsread
    cdef npy_intp _nbytes, _cbytes
    cdef npy_intp nhits, limit, skip
    cdef npy_intp expectedlen
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:870:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef object _cparams, _dflt
    cdef object _dtype
    cdef public object chunks
    cdef object _rootdir, datadir, metadir, _mode
    cdef object _attrs, iter_exhausted
    cdef ndarray iobuf, where_buf
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:878:9: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef object _rootdir, datadir, metadir, _mode
    cdef object _attrs, iter_exhausted
    cdef ndarray iobuf, where_buf
    # For block cache
    cdef int idxcache
    cdef ndarray blockcache
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:881:9: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        return self.len

    def __sizeof__(self):
        return self._cbytes

    cdef int getitem_cache(self, npy_intp pos, char *dest):
                                ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1755:33: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        # Safety check
        assert (nwrow == vlen)

    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
                       ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2158:24: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        # Safety check
        assert (nwrow == vlen)

    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
                                       ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2158:40: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        # Safety check
        assert (nwrow == vlen)

    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
                                                      ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2158:55: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        return lst

# This is the same than in utils.py, but works faster in extensions
cdef get_len_of_range(npy_intp start, npy_intp stop, npy_intp step):
    """Get the length of a (start, stop, step) range."""
    cdef npy_intp n
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:203:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    return n

cdef clip_chunk(npy_intp nchunk, npy_intp chunklen,
                npy_intp start, npy_intp stop, npy_intp step):
    """Get the limits of a certain chunk based on its length."""
    cdef npy_intp startb, stopb, blen, distance
        ^
------------------------------------------------------------

bcolz/carray_ext.pyx:214:9: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

    def __cinit__(self, object dobject, object atom, object cparams,
                  object _memory=True, object _compr=False):
        cdef int itemsize, footprint
        cdef size_t nbytes, cbytes, blocksize
        cdef dtype dtype_
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:307:13: 'dtype' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        return string

    cdef void _getitem(self, int start, int stop, char *dest):
        """Read data from `start` to `stop` and return it as a numpy array."""
        cdef int ret, bsize, blen, nitems, nstart
        cdef ndarray constants
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:463:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
            raise RuntimeError(
                "fatal error during Blosc decompression: %d" % ret)

    def __getitem__(self, object key):
        """__getitem__(self, key) -> values."""
        cdef ndarray array
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:489:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        """The directory for data files."""
        def __get__(self):
            return os.path.join(self.rootdir, DATA_DIR)

    def __cinit__(self, rootdir, metainfo=None, _new=False):
        cdef ndarray lastchunkarr
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:680:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        supported is on unidimensional arrays whose dtype is object (objects in
        composite dtypes are not supported).

        """
        cdef int itemsize, atomsize, chunksize
        cdef ndarray lastchunkarr
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1043:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        self.flush()

    def open_carray(self, shape, cparams, dtype, dflt,
                    expectedlen, cbytes, chunklen, xchunks=None):
        """Open an existing array."""
        cdef ndarray lastchunkarr
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1159:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    def open_carray(self, shape, cparams, dtype, dflt,
                    expectedlen, cbytes, chunklen, xchunks=None):
        """Open an existing array."""
        cdef ndarray lastchunkarr
        cdef object array_, _dflt
        cdef npy_intp calen
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1161:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
            self.resize(0)

    def fill_chunks(self, object array_):
        """Fill chunks, either in-memory or on-disk."""
        cdef int leftover, chunklen
        cdef npy_intp i, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1220:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

    def fill_chunks(self, object array_):
        """Fill chunks, either in-memory or on-disk."""
        cdef int leftover, chunklen
        cdef npy_intp i, nchunks
        cdef npy_intp nbytes, cbytes
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1221:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        """Fill chunks, either in-memory or on-disk."""
        cdef int leftover, chunklen
        cdef npy_intp i, nchunks
        cdef npy_intp nbytes, cbytes
        cdef chunk chunk_
        cdef ndarray remainder
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1223:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
            the carray.

        """
        cdef int atomsize, itemsize, chunksize, leftover
        cdef int nbytesfirst, chunklen, start, stop
        cdef npy_intp nbytes, cbytes, bsize, i, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1348:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        """
        cdef int atomsize, itemsize, chunksize, leftover
        cdef int nbytesfirst, chunklen, start, stop
        cdef npy_intp nbytes, cbytes, bsize, i, nchunks
        cdef ndarray remainder, arrcpy, dflts
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1349:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
            the object
            is enlarged instead.

        """
        cdef int atomsize, leftover, leftover2
        cdef npy_intp cbytes, bsize, nchunk2
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1455:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        -------
        reshaped_array : carray
            A copy of the original carray.

        """
        cdef npy_intp newlen, ilen, isize, osize, newsize, rsize, i
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1566:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        ------------
        out : NumPy scalar with `dtype`

        """
        cdef chunk chunk_
        cdef npy_intp nchunk, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1717:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        IMPORTANT: Any update operation (e.g. __setitem__) *must* disable this
        cache by setting self.idxcache = -2.
        """
        cdef int ret, atomsize, blocksize, offset
        cdef int idxcache, posinbytes, blocklen
        cdef npy_intp nchunk, nchunks, chunklen
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1772:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        eval

        """

        cdef int chunklen
        cdef npy_intp startb, stopb
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1867:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        """

        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, keychunk, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1868:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        """

        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, keychunk, nchunks
        cdef npy_intp nwrow, blen
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1869:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, keychunk, nchunks
        cdef npy_intp nwrow, blen
        cdef ndarray arr1
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:1870:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        --------
        eval

        """
        cdef int chunklen
        cdef npy_intp startb, stopb
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2017:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        eval

        """
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, keychunk, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2018:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

        """
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, keychunk, nchunks
        cdef npy_intp nwrow, blen, vlen
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2019:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        assert (nwrow == vlen)

    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
        cdef int chunklen
        cdef npy_intp startb, stopb
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2160:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nwrow, stop, cblen
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2161:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    # This is a private function that is specific for `eval`
    def _getrange(self, npy_intp start, npy_intp blen, ndarray out):
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nwrow, stop, cblen
        cdef npy_intp schunk, echunk, nchunk, nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2162:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

    cdef void bool_update(self, boolarr, value):
        """Update self in positions where `boolarr` is true with `value`
        array."""
        cdef int chunklen
        cdef npy_intp startb, stopb
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2205:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef void bool_update(self, boolarr, value):
        """Update self in positions where `boolarr` is true with `value`
        array."""
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, nchunks, nrows
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2206:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        """Update self in positions where `boolarr` is true with `value`
        array."""
        cdef int chunklen
        cdef npy_intp startb, stopb
        cdef npy_intp nchunk, nchunks, nrows
        cdef npy_intp nwrow, blen, vlen, n
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2207:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
            raise StopIteration  # end of iteration

    cdef int check_zeros(self, object barr):
        """Check for zeros.  Return 1 if all zeros, else return 0."""
        cdef int bsize
        cdef npy_intp nchunk
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2524:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
    cdef int check_zeros(self, object barr):
        """Check for zeros.  Return 1 if all zeros, else return 0."""
        cdef int bsize
        cdef npy_intp nchunk
        cdef carray carr
        cdef ndarray ndarr
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2526:13: 'ndarray' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        (__settitem__(), append()) in persistence mode.  If you don't do this,
        you risk loosing part of your modifications.

        """
        cdef chunk chunk_
        cdef npy_intp nchunks
            ^
------------------------------------------------------------

bcolz/carray_ext.pyx:2569:13: 'npy_intp' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...

# Initialization code

# The numpy API requires this function to be called before
# using any numpy facilities in an extension module.
import_array()
           ^
------------------------------------------------------------

bcolz/carray_ext.pyx:110:12: undeclared name not builtin: import_array

Error compiling Cython file:
------------------------------------------------------------
...

    # step corrections
    if step > 1:
        # Just correcting startb is enough
        distance = (nchunk * chunklen + startb) - start
        if distance % step > 0:
                   ^
------------------------------------------------------------

bcolz/carray_ext.pyx:231:20: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(carray_ext.pyx:9:0)
StatListNode.stats[39] = StatListNode(carray_ext.pyx:211:5)
StatListNode.stats[0] = CFuncDefNode(carray_ext.pyx:211:5,
    args = [...]/5,
    doc = u'Get the limits of a certain chunk based on its length.',
    modifiers = [...]/0,
    visibility = u'private')
File 'Nodes.py', line 392, in analyse_expressions: StatListNode(carray_ext.pyx:213:4,
    is_terminator = True)
File 'Nodes.py', line 5499, in analyse_expressions: IfStatNode(carray_ext.pyx:228:4)
File 'Nodes.py', line 5542, in analyse_expressions: IfClauseNode(carray_ext.pyx:228:7)
File 'Nodes.py', line 392, in analyse_expressions: StatListNode(carray_ext.pyx:230:8)
File 'Nodes.py', line 5499, in analyse_expressions: IfStatNode(carray_ext.pyx:231:8)
File 'Nodes.py', line 5541, in analyse_expressions: IfClauseNode(carray_ext.pyx:231:11)
File 'ExprNodes.py', line 450, in analyse_temp_boolean_expression: PrimaryCmpNode(carray_ext.pyx:231:27,
    operator = u'>',
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 10542, in analyse_types: PrimaryCmpNode(carray_ext.pyx:231:27,
    operator = u'>',
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 9173, in analyse_types: ModNode(carray_ext.pyx:231:20,
    infix = True,
    operator = u'%',
    result_is_used = True,
    use_managed_ref = True,
    zerodivision_check = True)
File 'ExprNodes.py', line 9742, in analyse_operation: ModNode(carray_ext.pyx:231:20,
    infix = True,
    operator = u'%',
    result_is_used = True,
    use_managed_ref = True,
    zerodivision_check = True)

Compiler crash traceback from this point on:
  File "/home/esc/anaconda/lib/python2.7/site-packages/Cython/Compiler/ExprNodes.py", line 9742, in analyse_operation
    self.cdivision = env.directives['cdivision'] or not self.type.signed
AttributeError: 'ErrorType' object has no attribute 'signed'
building 'bcolz.carray_ext' extension
C compiler: gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

compile options: '-DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -I/home/esc/anaconda/lib/python2.7/site-packages/numpy/core/include -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-r119 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/home/esc/anaconda/include/python2.7 -c'
extra options: '-msse2'
gcc: c-blosc/internal-complibs/zlib-1.2.8/compress.c
gcc: c-blosc/blosc/blosclz.c
gcc: c-blosc/blosc/blosc.c
In file included from c-blosc/blosc/blosc.c:24:0:
c-blosc/internal-complibs/lz4-r119/lz4.h:179:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
In file included from c-blosc/blosc/blosc.c:24:0:
c-blosc/internal-complibs/lz4-r119/lz4.h:237:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
gcc: c-blosc/internal-complibs/zlib-1.2.8/crc32.c
gcc: c-blosc/internal-complibs/lz4-r119/lz4hc.c
In file included from c-blosc/internal-complibs/lz4-r119/lz4hc.c:142:0:
c-blosc/internal-complibs/lz4-r119/lz4.h:179:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
In file included from c-blosc/internal-complibs/lz4-r119/lz4hc.c:142:0:
c-blosc/internal-complibs/lz4-r119/lz4.h:237:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
gcc: c-blosc/internal-complibs/snappy-1.1.1/snappy-sinksource.cc
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc: c-blosc/internal-complibs/zlib-1.2.8/trees.c
gcc: c-blosc/internal-complibs/zlib-1.2.8/infback.c
gcc: c-blosc/internal-complibs/snappy-1.1.1/snappy-c.cc
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc: c-blosc/internal-complibs/snappy-1.1.1/snappy-stubs-internal.cc
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc: c-blosc/blosc/shuffle.c
gcc: c-blosc/internal-complibs/zlib-1.2.8/gzclose.c
gcc: c-blosc/internal-complibs/zlib-1.2.8/zutil.c
gcc: bcolz/carray_ext.c
bcolz/carray_ext.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
bcolz/carray_ext.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
error: Command "gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -I/home/esc/anaconda/lib/python2.7/site-packages/numpy/core/include -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-r119 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/zlib-1.2.8 -I/home/esc/anaconda/include/python2.7 -c bcolz/carray_ext.c -o build/temp.linux-x86_64-2.7/bcolz/carray_ext.o -msse2" failed with exit status 1
grenoya commented 10 years ago

I have no such compilation problem here (once Cython version problem fixed). If it can be of some help:

esc commented 10 years ago

I am also using Cython 0.21 and the error is because Cython seems unable to find the bcolz/definitions.pxd file. If I add the path bcolz to the includes, it seems to work fine, also works with Cython 0.20.2.

esc commented 10 years ago

Here is the whole setup:

bcolz version:     0.7.2.dev
bcolz git info:    v0.7.1-34-gfbe30e666d
NumPy version:     1.9.0
Blosc version:     1.4.1 ($Date:: 2014-07-08 #$)
Blosc compressors: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib']
Numexpr version:   2.3.1
Python version:    2.7.8 |Anaconda 2.0.1 (64-bit)| (default, Aug 21 2014, 18:22:21) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Platform:          linux2-x86_64
Byte-ordering:     little
Detected cores:    4
esc commented 10 years ago

FWIW: I also have the same issue on the command line, and this can be fixed by using:

cython --include-dir bcolz/ bcolz/carray_ext.pyx
ilanschnell commented 10 years ago

I've noticed that problem too, with Cython 0.21. I think the problem is that from Cython.Compiler.Main import Version fails. Why not simply use from Cython import __version__ ?

esc commented 10 years ago

Pull-request is in progress at #53

ilanschnell commented 10 years ago

Thanks, that fixes it.

esc commented 10 years ago

Fixed with #53