ncsa / Genomics_EpiQuant

0 stars 0 forks source link

Output File Refactors + AWS Scripts #14

Closed sstevenshang closed 6 years ago

sstevenshang commented 6 years ago

This in PR:

  1. Refactored arg parser of SPMAEL module.
  2. Added various helper functions to help write/delete files from HDFS (adapted to AWS S3 as well).
  3. Added scripts to automate AWS operations (see aws/deploy_to_s3.sh and aws/run_in_emr.sh).

This should close the following issues:

Note:

sstevenshang commented 6 years ago

Sounds good. I made some updates:

I've tested and verified that this works on local machines. Should be ready to merge