Create a pipeline module to control

[skasberger]

Purpose:

• implement functions to handle mass of data
• allow easy control of different pipeline steps
• test the module

Functionalities:

• Control the workflow with a JSON file
  • Schema
• Work out exception handling
• Data Types
  • Dataverses
  • import_dataverses(list(dict))
  • get_dataverses(list(dict))
  • create_dataverses(list(dict))
  • create_dataverses_json(list(dict))
  • publish_dataverses(list(dict))
  • update_dataverses(list(dict))
  • delete_dataverses(list(dict))
  • Datasets
  • import_datasets(list(dict))
  • get_datasets(list(dict))
  • create_datasets(list(dict))
  • create_datasets_json(list(dict))
  • publish_datasets(list(dict))
  • update_datasets(list(dict))
  • delete_datasets(list(dict))
  • destroy_datasets(list(dict))
  • create_private_urls(list(dict))
  • Datafiles
  • import_datafiles(list(dict))
  • get_datafiles(list(dict))
  • upload_datafiles(list(dict))
  • create_datafiles_json(list(dict))
  • publish_datafiles(list(dict))
  • update_datafiles(list(dict))
  • delete_datafiles(list(dict))
  • replace_datafiles(list(dict))
  • Functions
  • from_CSV() #11
  • to_CSV() #11

[skasberger]

This will not be part of pyDataverse, as it can be done better with other tools (e. g. Apache Airflow).