search

civisanalytics / civis-r

Civis API Client for R: https://civisanalytics.com/products/civis-platform/
Other
16 stars 12 forks source link

Switch to CSVImports for Redshift upload #218

Open patr1ckm opened 4 years ago

patr1ckm commented 4 years ago

In general:

  1. Add a utility get_headers.
  2. Switch from imports_post to imports_post_files_csv in write_civis.character and write_civis.numeric.
  3. Before uploading, files need to be cleaned withfiles_post_process_csv. Note in the reference PR the steps that are needed to process the cleaning results.
  4. The client will need to be regenerated to access the new endpoints.
  5. Support multiple file ids in write_civis.numeric. If multiple files are uploaded, check that the schema, headers, delimiter, and compression are consistent across files. Upload the files in parallel.
  6. Document the upsert option to if_exists.

See https://github.com/civisanalytics/civis-python/pull/328 for all details.