search

hellonarrativ / spectrify

Export Redshift data and convert to Parquet for use with Redshift Spectrum or other data warehouses.
https://aws.amazon.com/blogs/big-data/narrativ-is-helping-producers-monetize-their-digital-content-with-amazon-redshift/
MIT License
116 stars 25 forks source link

CsvConverter: Support CSV basic options. #27

Closed nirmo-lightricks closed 6 years ago

nirmo-lightricks commented 6 years ago

Support different delimiters, escaping and quoting string.

nirmo-lightricks commented 6 years ago

Hi @c-nichols, I made all the changes, but for some reason I get the following error (see travis log for Python 3.4):

    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-viw7b8i2/pyarrow/setup.py", line 29, in <module>
        from Cython.Distutils import build_ext as _build_ext
    ImportError: No module named 'Cython'

do you know how to solve it? Thanks :)

c-nichols commented 6 years ago

Thanks Nir! I will take a look... I'm not sure off the top of my head. Possible that the install process for Arrow has changed.

c-nichols commented 6 years ago

@nirmo-lightricks if you rebase master, the tests should all pass (dropped support for Python 3.4)

nirmo-lightricks commented 6 years ago

@c-nichols done :)