KitWallace / AIDVIEW-DB

A repository, browser and API for IATI activities
1 stars 0 forks source link

Project value apportionment in multi-valued facets #111

Closed KitWallace closed 11 years ago

KitWallace commented 11 years ago

Project value is apportioned over the multiple occurrences of say country. However in aggregations, this apportionment was not used

KitWallace commented 11 years ago

Change olap to compute apportioned project values:

This requires

Change api to do this computation on computed aggregations -- this means changing the api configuration to use the olap configuration file

KitWallace commented 11 years ago

olap and api modules updated - as is the olap conf.xml file

KitWallace commented 11 years ago

Problem remains with multi-level grouping: In the malaria case, the activities are selected by sector and then grouped by country, and the country value summed, but if the sector is apportioned as well, then each value should be weighted by the sector proportion. If there were multiple sectors, then the proportion of each sector would be summed - This is significantly more complicated and slower than the simple approach especially when data-driven. This will need further design work to achieve but for accurate value reporting this apportionment is necessary.

The problem is that if apportionment is only done at the top level in the olap, unless this apportionment is also done all the way down complex selections, sub-selections may appear to have a higher value than the total for a selection.

Until this is thought through further, I have reverted to the un-proportioned values but show both in the facet summaries

KitWallace commented 11 years ago

This is now fully implemented in the query interface using a generated expression. This runs rather slower than the previous version so its a trade-off between accuracy and speed. On balance given that there is a caching layer, it would be better to have the right data I think.

See http://data.aidinfolabs.org/xquery/woapi2.xq for this new computation, showing both simple summed values and apportioned values

KitWallace commented 11 years ago

Some further optimisation of the query results in performance which is only 25% slower so this more accurate approach will be put in place.

KitWallace commented 11 years ago

Having heard nothing to the contrary I've now made this change to the live system - I would welcome some feedback.

Mattdevinit commented 11 years ago

thanks for doing this - would be good to go through this if/when we meet up in Bristol