search

go-sif / sif

Sif is a framework for fast, predictable, general-purpose distributed computing in the map/reduce paradigm.
Apache License 2.0
32 stars 3 forks source link

Implementation of accumulators for Sif, for customizable reduction #10

Closed Ghnuberath closed 4 years ago

Ghnuberath commented 4 years ago

Accumulators offer full customization of reduction technique, in exchange for losing the ability to further transform and reduce data.

Node.Run() signature was altered to permit the return of an Accumulator, in addition to CollectedPartitions.

codecov-io commented 4 years ago

Codecov Report

Merging #10 into master will increase coverage by 0.19%. The diff coverage is 52.57%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #10      +/-   ##
==========================================
+ Coverage   55.34%   55.53%   +0.19%     
==========================================
  Files          70       72       +2     
  Lines        4315     4633     +318     
==========================================
+ Hits         2388     2573     +185     
- Misses       1666     1777     +111     
- Partials      261      283      +22
Impacted Files Coverage Δ
cluster/node.go 50.00% <ø> (ø)
internal/dataframe/task.go 100.00% <ø> (ø)
internal/rpc/s_execution.pb.go 42.44% <12.50%> (-1.26%) :arrow_down:
internal/rpc/s_partition.pb.go 36.49% <27.69%> (+3.73%) :arrow_up:
internal/dataframe/dataframe-executable.go 79.16% <45.45%> (-10.31%) :arrow_down:
cluster/s_partition.go 68.85% <65.51%> (-1.05%) :arrow_down:
cluster/coordinator.go 70.22% <70.68%> (-1.21%) :arrow_down:
internal/dataframe/plan_executor.go 81.96% <88.88%> (+0.75%) :arrow_up:
accumulators/count.go 89.47% <89.47%> (ø)
cluster/worker.go 70.90% <100.00%> (ø)
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 81bf579...3cddb90. Read the comment docs.