Measuring impact and performance

[ScatteredInk]

I had an initial go at how we might want to model the numbers part of our specification (a metric attached to an organisation, place or project at a particular point-in-time point or time-frame), but think this may have been a case of re-inventing the wheel. We might want to think about JSONStat as an alternative. We need to able to:

• represent our numbers
• contextualise them
• say what kind of numbers they are, so that people use them right (median wage, absolute value of capital returned etc)

I think JSONStat does all this.

[BobHarper1]

I think this is a good approach, and well found.

Some notes, for future reference

We need to able to:

• represent our numbers

Dimensions with a metric role indicate the actual measured numbers, which are represented in the value array. And it can also add symbols e.g. £

• contextualise them

Dimensions can carry a descriptive label, and a note can be applied to the dataset, dimension or category

• say what kind of numbers they are, so that people use them right (median wage, absolute value of capital returned etc)

I think the approach would be to use the labels, with notes. I don't think there is a structured way to do so codified in the schema.

Another way this could be used: to attached a collection of related datasets http://json-stat.org/samples/collection.json

[BobHarper1]

IATI Standard result element could give us some pointers on how to do this (h/t @timgdavies for the suggestion)

The IATI activity standard enables the publishing of information on measurable results from an iati-activity through use of the result element via:

• result - a container for reporting outputs, outcomes, impacts and other results that stem directly from the iati-activity. ...

  Considerations

• Every result should include an indicator, which in turn details a period of time, and then a target and actual measure. ...
• An indicator can be repeated within any result dataset.
• It is also recommended to include a baseline measure for each result recorded.
• Any target, actual and baseline can have attached a comment to provide additional narrative or information. ... http://iatistandard.org/202/activity-standard/overview/result/

Some published data: http://d-portal.org/q.xml?aid=47045-PRK-M-UNICEF ... and how it's displayed in the IATI d-portal http://d-portal.org/ctrack.html?search&publisher=47045#view=act&aid=47045-PRK-M-UNICEF

[ScatteredInk]

From a discussion on bookkeeping, there may be a use case for overlapping metrics:

As an analyst, I want to access data that is more granular than, or different to, that stated in an organisation's accounts, so that I can assess its financial health.

Examples:

1. Different reporting requirements. Some funders require standardised reporting periods. Some analysts may prefer to standardise reporting periods.
2. Tangible assets. Buildings are rarely separated from other tangible assets but, to assess liquidity needs, an analyst may want to access figures on building valuations, cash-in-bank etc.

For case (1), the current method being discussed for metrics would allow us to attach multiple metrics with different periods. For case (2), where an analyst may need to perform calculations based on metrics within the same period, we would probably want to develop guidance and glossaries around what metrics can be combined in this way, and think about a source field in the metric definition.