search

hellonarrativ / spectrify

Export Redshift data and convert to Parquet for use with Redshift Spectrum or other data warehouses.
https://aws.amazon.com/blogs/big-data/narrativ-is-helping-producers-monetize-their-digital-content-with-amazon-redshift/
MIT License
116 stars 25 forks source link

ImportError: No module named 'Cython' #32

Closed nirmo-lightricks closed 6 years ago

nirmo-lightricks commented 6 years ago

Description

Hi, When I create a new PR it consistently fails on python3.4. getting this message:

https://files.pythonhosted.org/packages/c0/a0/f7e9dfd8988d94f4952f9b50eb04e14a80fbe39218520725aab53daab57c/pyarrow-0.10.0.tar.gz (2.1MB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-6xo5oanm/pyarrow/setup.py", line 29, in <module>
        from Cython.Distutils import build_ext as _build_ext
    ImportError: No module named 'Cython'

It's related to the setup of pyarrow.

What I Did

you can see full description in Travis job log: https://travis-ci.org/hellonarrativ/spectrify/jobs/413090776

c-nichols commented 6 years ago

I think pyarrow dropped support for python 3.4. While we could probably get away with pinning dependencies or adding 3.4-specific install routines, it might be easiest to drop support for python 3.4. Thoughts?

nirmo-lightricks commented 6 years ago

I did some investigation, I installed Cython manually (using the requirements_dev.txt) and then I got another exception, which is a known issue in setuptools_scm (https://github.com/pypa/setuptools_scm/issues/297). hope they will solve it soon.

If pyarrow is no longer supports 3.4 I would remove the support from python 3.4 as well. otherwise the project will use a legacy version of pyarrow. other approach is to deal with it and add manual support for python 3.4. I think this patch will work (once they fix the setuptools_scm issue) : https://github.com/hellonarrativ/spectrify/pull/35

c-nichols commented 6 years ago

I am going to drop support for python 3.4 later today

nirmo-lightricks commented 6 years ago

👍