HEPCloud / decisionengine

HEPCloud Decision Engine framework
Apache License 2.0
6 stars 25 forks source link

Add channel-combination facilities #524

Closed knoepfel closed 2 years ago

knoepfel commented 2 years ago

The facilities provided by this PR enable assembling channel configurations in a composable way. See the below example.

This PR also provides the ability to specify Jsonnet path directories when loading Jsonnet file.


Consider the two Jsonnet configurations:

A.libsonnet

{
  sources: {
    a1: { ... }
    a2: { parameters: { source_channel: ... }, ... }
  }
}

B.libsonnet

{                                                                                                                                                                                                        
  sources: {
    b: { ... }
  }
}

They can be combined via the following configuration:

Final.jsonnet

local de_std = import 'de_std.libjsonnet';
local channels = [import 'A.libsonnet', import 'B.libsonnet'];
{
  sources: de_std.sources_from(channels)
}

The final JSON configuration would look like:

{
  "sources": {
    "a1": { ... },
    "b": { ... }
  }
}

Note that the source-proxy configurations are not persisted.

codecov[bot] commented 2 years ago

Codecov Report

Merging #524 (67780cc) into master (a4a7938) will increase coverage by 0.01%. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #524      +/-   ##
==========================================
+ Coverage   83.39%   83.40%   +0.01%     
==========================================
  Files          47       47              
  Lines        3089     3091       +2     
  Branches      433      434       +1     
==========================================
+ Hits         2576     2578       +2     
  Misses        473      473              
  Partials       40       40              
Flag Coverage Δ
python-3.6 82.86% <100.00%> (+0.01%) :arrow_up:
python-3.9 82.99% <100.00%> (-0.03%) :arrow_down:
python-pypy-3.7 82.99% <100.00%> (+0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/decisionengine/framework/config/ValidConfig.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a4a7938...67780cc. Read the comment docs.

jcpunk commented 2 years ago

This looks good to me.