search

Green-Software-Foundation / if

Impact Framework
https://if.greensoftware.foundation/
MIT License
149 stars 40 forks source link

Multiple aggregation not working as intended. #1047

Open mouhamadalmounayar opened 1 week ago

mouhamadalmounayar commented 1 week ago

Description of the Error

Aggregation feature is not working as intended when multiple metrics are provided even though no error is being thrown. (I opened this issue since I found that it is not working as intended even on the main branch.)

Expected Behaviour

We should get an aggregated field with all metrics values properly computed.

Actual Behaviour

Only the first metric is being computed. And its value is being copied from the last output only.

Steps to Reproduce

Run the following manifest file with the main branch version.

Manifest File That Generated the Error

name: carbon_pipeline
aggregation:
  metrics:
  - carbon
  - energy
  type: time
initialize:
  plugins:
    teads-curve:
      method: TeadsCurve
      path: '@grnsft/if-unofficial-plugins'
    multiply:
      method: Multiply
      path: builtin
      config:
        input-parameters:
        - resources-reserved
        - cpu/energy
        output-parameter: cpu/energy-total
    e-mem:
      method: EMem
      path: '@grnsft/if-plugins'
    sci-e:
      method: SciE
      path: '@grnsft/if-plugins'
    sci-m:
      method: SciM
      path: '@grnsft/if-plugins'
    sci-o:
      method: SciO
      path: '@grnsft/if-plugins'
    sci:
      method: Sci
      path: '@grnsft/if-plugins'
      config: 
        functional-unit: R
  outputs:
  - yaml
tree:
  defaults:
    memory/utilization: 100
    device/emissions-embodied: 1887000
    cpu/thermal-design-power: 3.76
    device/expected-lifespan: 126230400
    duration: 1800
  pipeline:
    compute:
    - teads-curve
    - multiply
    - e-mem
    - sci-e
    - sci-m
    - sci-o
    - sci
  children:
    0:
      children:
        child_1
          inputs:
          - timestamp: 1727958600
            cpu/utilization: 35.32553725926037
            grid/carbon-intensity: 251.0
            resources-reserved: 1.0
            resources-total: 90
            memory/capacity: 2.576980378
          - timestamp: 1727960400
            cpu/utilization: 33.50961455555754
            grid/carbon-intensity: 251.0
            resources-reserved: 1.0
            resources-total: 90
            memory/capacity: 2.576980378
          - timestamp: 1727962200
            cpu/utilization: 34.02016962963023
            grid/carbon-intensity: 251.0
            resources-reserved: 1.0
            resources-total: 90
            memory/capacity: 2.576980378

The output that has been generated : 

name: carbon_pipeline
aggregation:
  metrics:
    - carbon
    - energy
  type: time
initialize:
  plugins:
    teads-curve:
      path: '@grnsft/if-unofficial-plugins'
      method: TeadsCurve
    multiply:
      path: builtin
      method: Multiply
      config:
        input-parameters:
          - resources-reserved
          - cpu/energy
        output-parameter: cpu/energy-total
    e-mem:
      path: '@grnsft/if-plugins'
      method: EMem
    sci-e:
      path: '@grnsft/if-plugins'
      method: SciE
    sci-m:
      path: '@grnsft/if-plugins'
      method: SciM
    sci-o:
      path: '@grnsft/if-plugins'
      method: SciO
    sci:
      path: '@grnsft/if-plugins'
      method: Sci
      config:
        functional-unit: R
execution:
  command: >-
    C:\Users\malmounayar\programs\node-v20.17.0-win-x64\node.exe
    C:\Users\malmounayar\if\build\if-run\index.js --manifest
    ..\GreenIt\test-manifest\inputs\multiple-aggregation.yaml --output
    ..\GreenIt\test-manifest\outputs\multiple-aggregation.yaml
  environment:
    if-version: 0.7.0
    os: "Microsoft Windows 11 Enterprise\r"
    os-version: 10.0.22631 N/A Build 22631
    node-version: 20.17.0
    date-time: 2024-10-10T16:07:10.065Z (UTC)
    dependencies:
      - '@azure/abort-controller@1.1.0 extraneous -> undefined'
      - '@azure/arm-compute@21.6.0 extraneous -> undefined'
      - '@azure/arm-monitor@7.0.0 extraneous -> undefined'
      - '@azure/core-auth@1.8.0 extraneous -> undefined'
      - '@azure/core-client@1.9.2 extraneous -> undefined'
      - '@azure/core-lro@2.7.2 extraneous -> undefined'
      - '@azure/core-paging@1.6.2 extraneous -> undefined'
      - '@azure/core-rest-pipeline@1.17.0 extraneous -> undefined'
      - '@azure/core-tracing@1.2.0 extraneous -> undefined'
      - '@azure/core-util@1.10.0 extraneous -> undefined'
      - '@azure/identity@4.4.1 extraneous -> undefined'
      - '@azure/logger@1.1.4 extraneous -> undefined'
      - '@azure/msal-browser@3.26.1 extraneous -> undefined'
      - '@azure/msal-common@14.15.0 extraneous -> undefined'
      - '@azure/msal-node@2.15.0 extraneous -> undefined'
      - '@babel/core@7.22.10'
      - '@babel/preset-typescript@7.23.3'
      - '@cloud-carbon-footprint/aws@0.15.0 extraneous -> undefined'
      - '@cloud-carbon-footprint/common@1.15.0 extraneous -> undefined'
      - '@cloud-carbon-footprint/core@0.17.3 extraneous -> undefined'
      - '@commitlint/cli@18.6.0'
      - '@commitlint/config-conventional@18.6.0'
      - '@google-cloud/common@3.10.0 extraneous -> undefined'
      - '@google-cloud/iam-credentials@1.2.0 extraneous -> undefined'
      - '@google-cloud/logging-winston@4.2.4 extraneous -> undefined'
      - '@google-cloud/logging@9.9.0 extraneous -> undefined'
      - '@google-cloud/paginator@3.0.7 extraneous -> undefined'
      - '@google-cloud/projectify@2.1.1 extraneous -> undefined'
      - '@google-cloud/promisify@2.0.4 extraneous -> undefined'
      - '@grnsft/if-core@0.0.25'
      - '@grnsft/if-plugins@v0.3.2 extraneous -> undefined'
      - '@grnsft/if-unofficial-plugins@v0.3.1 extraneous -> undefined'
      - '@grpc/grpc-js@1.6.12 extraneous -> undefined'
      - '@grpc/proto-loader@0.6.13 extraneous -> undefined'
      - '@jest/globals@29.7.0'
      - '@protobufjs/aspromise@1.1.2 extraneous -> undefined'
      - '@protobufjs/base64@1.1.2 extraneous -> undefined'
      - '@protobufjs/codegen@2.0.4 extraneous -> undefined'
      - '@protobufjs/eventemitter@1.1.0 extraneous -> undefined'
      - '@protobufjs/fetch@1.1.0 extraneous -> undefined'
      - '@protobufjs/float@1.0.2 extraneous -> undefined'
      - '@protobufjs/inquire@1.1.0 extraneous -> undefined'
      - '@protobufjs/path@1.1.2 extraneous -> undefined'
      - '@protobufjs/pool@1.1.0 extraneous -> undefined'
      - '@protobufjs/utf8@1.1.0 extraneous -> undefined'
      - '@sinonjs/samsam@8.0.2 extraneous -> undefined'
      - '@sinonjs/text-encoding@0.7.3 extraneous -> undefined'
      - '@tgwf/co2@0.14.4 extraneous -> undefined'
      - '@tootallnate/once@2.0.0 extraneous -> undefined'
      - '@types/jest@29.5.8'
      - '@types/js-yaml@4.0.9'
      - '@types/long@4.0.2 extraneous -> undefined'
      - '@types/luxon@3.4.2'
      - '@types/node@20.9.0'
      - '@types/ramda@0.29.12 extraneous -> undefined'
      - '@types/tgwf__co2@0.0.0 extraneous -> undefined'
      - abort-controller@3.0.0 extraneous -> undefined
      - aws-sdk-mock@5.9.0 extraneous -> undefined
      - aws-sdk@2.1691.0 extraneous -> undefined
      - axios-mock-adapter@1.22.0
      - axios@1.7.2
      - bignumber.js@9.1.2 extraneous -> undefined
      - bluebird@3.7.2 extraneous -> undefined
      - buffer-equal-constant-time@1.0.1 extraneous -> undefined
      - copyfiles@2.4.1 extraneous -> undefined
      - core-util-is@1.0.3 extraneous -> undefined
      - cross-env@7.0.3
      - csv-parse@5.5.6
      - csv-stringify@6.4.6
      - csvtojson@2.0.10 extraneous -> undefined
      - diff@5.2.0 extraneous -> undefined
      - dotenv@16.4.5 extraneous -> undefined
      - duplexify@4.1.3 extraneous -> undefined
      - ecdsa-sig-formatter@1.0.11 extraneous -> undefined
      - ee-first@1.1.1 extraneous -> undefined
      - end-of-stream@1.4.4 extraneous -> undefined
      - ent@2.2.1 extraneous -> undefined
      - event-target-shim@5.0.1 extraneous -> undefined
      - eventid@2.0.1 extraneous -> undefined
      - events@3.3.0 extraneous -> undefined
      - extend@3.0.2 extraneous -> undefined
      - fast-text-encoding@1.0.6 extraneous -> undefined
      - fixpack@4.0.0
      - gaxios@4.3.3 extraneous -> undefined
      - gcp-metadata@4.3.1 extraneous -> undefined
      - google-auth-library@7.14.1 extraneous -> undefined
      - google-gax@2.30.5 extraneous -> undefined
      - google-p12-pem@3.1.4 extraneous -> undefined
      - gtoken@5.3.2 extraneous -> undefined
      - gts@5.2.0
      - husky@8.0.3
      - is-generator-function@1.0.10 extraneous -> undefined
      - is-stream-ended@0.1.4 extraneous -> undefined
      - is-utf8@0.2.1 extraneous -> undefined
      - jest@29.7.0
      - jmespath@0.16.0 extraneous -> undefined
      - js-yaml@4.1.0
      - json-bigint@1.0.0 extraneous -> undefined
      - jsonwebtoken@9.0.2 extraneous -> undefined
      - just-extend@6.2.0 extraneous -> undefined
      - jwa@2.0.0 extraneous -> undefined
      - jws@4.0.0 extraneous -> undefined
      - lint-staged@15.2.2
      - lodash.get@4.4.2 extraneous -> undefined
      - lodash.includes@4.3.0 extraneous -> undefined
      - lodash.isboolean@3.0.3 extraneous -> undefined
      - lodash.isinteger@4.0.4 extraneous -> undefined
      - lodash.isnumber@3.0.3 extraneous -> undefined
      - lodash.mapvalues@4.6.0 extraneous -> undefined
      - lodash.once@4.1.1 extraneous -> undefined
      - long@4.0.0 extraneous -> undefined
      - luxon@3.4.4
      - mkdirp@1.0.4 extraneous -> undefined
      - moment@2.30.1 extraneous -> undefined
      - nise@5.1.9 extraneous -> undefined
      - node-forge@1.3.1 extraneous -> undefined
      - noms@0.0.0 extraneous -> undefined
      - object-hash@3.0.0 extraneous -> undefined
      - on-finished@2.4.1 extraneous -> undefined
      - path-to-regexp@6.3.0 extraneous -> undefined
      - process-nextick-args@2.0.1 extraneous -> undefined
      - proto3-json-serializer@0.1.9 extraneous -> undefined
      - protobufjs@6.11.3 extraneous -> undefined
      - pump@3.0.2 extraneous -> undefined
      - pumpify@2.0.1 extraneous -> undefined
      - querystring@0.2.0 extraneous -> undefined
      - ramda@0.29.1 extraneous -> undefined
      - release-it@16.3.0
      - retry-request@4.2.2 extraneous -> undefined
      - rimraf@5.0.5
      - sax@1.2.1 extraneous -> undefined
      - sinon@17.0.1 extraneous -> undefined
      - stoppable@1.1.0 extraneous -> undefined
      - stream-events@1.0.5 extraneous -> undefined
      - stream-shift@1.0.3 extraneous -> undefined
      - stubs@3.0.0 extraneous -> undefined
      - teeny-request@7.2.0 extraneous -> undefined
      - through2@2.0.5 extraneous -> undefined
      - traverse@0.6.10 extraneous -> undefined
      - ts-command-line-args@2.5.1
      - ts-jest@29.1.1
      - ts-toolbelt@9.6.0 extraneous -> undefined
      - typedarray.prototype.slice@1.0.3 extraneous -> undefined
      - types-ramda@0.29.10 extraneous -> undefined
      - typescript-cubic-spline@1.0.1
      - typescript@5.2.2
      - url@0.10.3 extraneous -> undefined
      - util@0.12.5 extraneous -> undefined
      - uuid@8.3.2 extraneous -> undefined
      - winston@3.11.0
      - xml2js@0.6.2 extraneous -> undefined
      - xmlbuilder@11.0.1 extraneous -> undefined
      - xtend@4.0.2 extraneous -> undefined
      - zod@3.23.8
  status: success
tree:
  defaults:
    memory/utilization: 100
    device/emissions-embodied: 1887000
    cpu/thermal-design-power: 3.76
    device/expected-lifespan: 126230400
    duration: 1800
  pipeline:
    compute:
      - teads-curve
      - multiply
      - e-mem
      - sci-e
      - sci-m
      - sci-o
      - sci
  children:
    '0':
      children:
        child_1:
          inputs:
            - timestamp: 1727958600
              cpu/utilization: 35.32553725926037
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
            - timestamp: 1727960400
              cpu/utilization: 33.50961455555754
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
            - timestamp: 1727962200
              cpu/utilization: 34.02016962963023
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
          outputs:
            - timestamp: 1727958600
              cpu/utilization: 35.32553725926037
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
              memory/utilization: 100
              device/emissions-embodied: 1887000
              cpu/thermal-design-power: 3.76
              device/expected-lifespan: 126230400
              duration: 1800
              cpu/energy: 0.0012094032917143286
              cpu/energy-total: 0.0012094032917143286
              memory/energy: 0.001010176308176
              energy: 0.0022195795998903286
              carbon-embodied: 0.2989771085253632
              carbon-operational: 0.5571144795724725
              carbon: 0.00047560643783213096
              sci: 0.00047560643783213096
            - timestamp: 1727960400
              cpu/utilization: 33.50961455555754
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
              memory/utilization: 100
              device/emissions-embodied: 1887000
              cpu/thermal-design-power: 3.76
              device/expected-lifespan: 126230400
              duration: 1800
              cpu/energy: 0.001179441369467165
              cpu/energy-total: 0.001179441369467165
              memory/energy: 0.001010176308176
              energy: 0.002189617677643165
              carbon-embodied: 0.2989771085253632
              carbon-operational: 0.5495940370884345
              carbon: 0.00047142841422988756
              sci: 0.00047142841422988756
            - timestamp: 1727962200
              cpu/utilization: 34.02016962963023
              grid/carbon-intensity: 251
              resources-reserved: 1
              resources-total: 90
              memory/capacity: 2.576980378
              memory/utilization: 100
              device/emissions-embodied: 1887000
              cpu/thermal-design-power: 3.76
              device/expected-lifespan: 126230400
              duration: 1800
              cpu/energy: 0.0011880189188019983
              cpu/energy-total: 0.0011880189188019983
              memory/energy: 0.001010176308176
              energy: 0.002198195226977998
              carbon-embodied: 0.2989771085253632
              carbon-operational: 0.5517470019714775
              carbon: 0.00047262450583157817
              sci: 0.00047262450583157817
          aggregated:
            carbon: 0.00047262450583157817
zanete commented 1 day ago

@narekhovhannisyan please could you verify that this should no longer be an issue? 🙏