projectglow / glow

An open-source toolkit for large-scale genomic analysis
https://projectglow.io
Apache License 2.0
262 stars 107 forks source link

Add ganglia image / layer deployment to Glow docker #489

Closed a0x8o closed 2 years ago

a0x8o commented 2 years ago

What changes are proposed in this pull request?

Added ganglia image / layer deployment to Glow docker to support ganglia metrics collection and reporting in Databricks Container Services

How is this patch tested?

Added an image / layer with a tested ganglia deployment and configuration designed specifically to perform metrics collection across the entire Spark cluster in Databricks Container Services. This implementation is in contained in the ./docker/databricks/dbr/dbr9.1/ganglia folder. Note that the ganglia layer must be built last.

Updated the ./docker/databricks/build.sh script to: 1) build the projectglow/databricks-glow image with the ganglia layer 2) build a new projectglow/databricks-glow-minus-ganglia image without the ganglia layer (for debugging and testing)

Updated the README.mdfile to reflect these changes and the rationale behind them

codecov[bot] commented 2 years ago

Codecov Report

Merging #489 (c1c87a4) into master (99e1075) will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #489   +/-   ##
=======================================
  Coverage   93.66%   93.66%           
=======================================
  Files          95       95           
  Lines        4875     4875           
  Branches      457      457           
=======================================
  Hits         4566     4566           
  Misses        309      309           

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 99e1075...c1c87a4. Read the comment docs.