arnegevaert / benchmark

MIT License
2 stars 0 forks source link

Extend result structure #129

Closed arnegevaert closed 3 years ago

arnegevaert commented 3 years ago

Result object should support nested metrics. This allows for:

Possible layout:

- deletion
  - logit
    - InputXGradient [128 x 50]
    - IntegratedGradients [128 x 50]
  - softmax
    - InputXGradient [128 x 50]
    - IntegratedGradients [128 x 50]
  - sigmoid
    - InputXGradient [128 x 50]
    - IntegratedGradients [128 x 50]
- infidelity
  - mse
    - InputXGradient [128 x 1]
    - IntegratedGradients [128 x 1]
  - corr
    - InputXGradient [128 x 1]
    - IntegratedGradients [128 x 1]

Also add metadata flag to indicate if results are "higher/lower = better" (inverted: bool)