arup-group / gelato

Gelato turns MATSim outputs into transport sustainability KPI metrics
GNU General Public License v3.0
10 stars 1 forks source link

Add scaling/normalisation of KPI outputs #77

Closed KasiaKoz closed 7 months ago

KasiaKoz commented 8 months ago

Resolves: https://github.com/arup-group/gelato/issues/69 Fixes: https://github.com/arup-group/gelato/issues/73

This PR makes updates to KPI outputs to give a scaled/normalised value alongside the actual KPI value. The output types of the methods change, majority of them are now maps with two keys: "actual" and "normalised". They are still saved to csv, now with two columns: "actual" and "normalised" (examples below for the integration test files).

[Bonus] When working on this, adding tests specifically, I found a bug in GHG KPI and fixed it right there and then.

Normalisation

We normalise the KPI outputs using an affine transformation. It maps linearly the given interval (unique to each KPI) to [0,10]. I made Normaliser an interface, for if/when we want to change the type of transformation. The bounds of the mapped interval ([0,10]) were deliberately kept changeable - this made it easier to include the normaliser in tests in a simpler capacity (multiplicative constant). Thinking about it now, maybe I should have mocked this, the linear normaliser is tested separately - let me know what you think. I think being able to change scale bounds is still useful if one day we decide we want to map to [0,1] for example.

LinearNormaliser defaults to [0,10] right now, but I still wanted to specify the mapped interval in MatsimKpiGenerator. Example:

LinearNormaliser normaliser = new LinearNormaliser(0, 10, 0, 100)

maps values between 0 and 100 to values between 0 and 10.

One funny thing you will notice is that sometimes the interval for a KPI is flipped, big values are bad and small values are good. This is handled by the same normaliser (using factors of -1 under the hood) and the normaliser object is specified like this:

LinearNormaliser normaliser = new LinearNormaliser(0, 10, 100, 0)

i.e. mapping values between 100 and 0 to values between 0 and 10.

Let me know if that API is clear, I think of it as [0, 10] -> [100, 0]. I briefly considered a dedicated class for the reverse normaliser, but I thought it might look more confusing new ReverseLinearNormaliser normaliser = ReverseLinearNormaliser(0, 10, 0, 100), because you need to pay attention to the name of the class as well as the interval values. That class would be almost identical to the LinearNormaliser class too, with only a couple of small changes. I think what I've got here fells more natural but keen to get some opinions on that.

Tests

Majority of the new code is tests for the KPIs that were being normalised. The more complicated KPIs get more tests. This also meant more content being added to the builders that set up the inputs to produce different behaviour in the tests. Very happy to be plugging those holes.

Integration tests

Outputs that are benchmarks for integration tests changed for all of the KPIs being scaled. Below are the details.

smol

KPI old outputs (content of csv file) new outputs (content of csv file)
Affordability 0.34
actual normalised
0.34 10.0
PT Wait Time 601.0 (~10min)
actual normalised
601.0 4.98
Occupancy. 0.17
actual normalised
0.17 0.0
GHG 1.61 (9.67 after bug fix)
actual normalised
9.67 0.0
Travel Time 33.31818181818182
actual normalised
33.32 7.09
Access to Mobility services (bus) 11.11
actual normalised
11.11 1.11
Access to Mobility services (rail) 0.0
actual normalised
0.0 0.0
Access to Mobility services (used PT) 0.0
actual normalised
0.0 0.0
Congestion
mode Mean [delayRatio]
bus 2.83
car 3.57
mode Mean [delayRatio] Normalised [Mean [delayRatio]]
bus 2.83 0.95
car 3.57 0.0

drt

KPI old outputs (content of csv file) new outputs (content of csv file)
Affordability 1.98
actual normalised
1.98 0.0
PT Wait Time 657.0 (~11mins)
actual normalised
657.0 4.05
Occupancy. 0.09
actual normalised
0.09 0.0
GHG 5.1 (35.7 after bug fix)
actual normalised
35.7 0.0
Travel Time 46.75
actual normalised
46.75 5.41
Access to Mobility services (bus) 0.0
actual normalised
0.0 0.0
Access to Mobility services (rail) 0.0
actual normalised
0.0 0.0
Access to Mobility services (used PT) 11.11
actual normalised
11.11 1.11
Congestion
mode Mean [delayRatio]
bus 2.83
drt 6.24
mode Mean [delayRatio] Normalised [Mean [delayRatio]]
bus 2.83 0.95
car 6.24 0.0
KasiaKoz commented 7 months ago

A few somewhat small changes: