search

PHACDataHub / data-mesh-ref-impl

Data Mesh Reference Implementation with standalone example use cases
Apache License 2.0
6 stars 3 forks source link

Refactoring the data upload process #57

Closed simardeep1792 closed 10 months ago

simardeep1792 commented 10 months ago

Quick Overview of the changes:

  1. Improved Parallelism: Leveraged GNU Parallel to increase concurrent uploading, dramatically improving the speed of data transfers.
  2. Streamlined Dependencies: Implemented checks for required tools (jq, parallel, and curl), auto-installing them if missing. This ensures a smoother and more user-friendly experience.
  3. Enhanced Validation: Strengthened input validation for province or territory abbreviations, ensuring correct data targeting.
  4. Simplified Upload Logic: Refined the upload function logic to clearly indicate upload successes and failures based on server responses.