search

pwyf / 2018-index-indicator-definitions

1 stars 0 forks source link

Annual Report #3

Open andylolz opened 7 years ago

andylolz commented 7 years ago

Description

Annual reports outline basic (normally aggregate) information about how aid was spent in the previous year, broken down by sector and/or country. This should be backward looking.

As previously, documents will be manually checked to verify that they contain the required information to score for the indicator.

Proposed test

`document-link/category[@code="B01"]` should be present
breidertmt commented 7 years ago

Based on the information this is a two part test correct? 1). B01 has to be present 2). Document coded B01 is then manually checked.

Do you get anything for having a B01 present, but then manually checked does not pass i.e. perhaps is not broken down by country.

andylolz commented 7 years ago

This is the same as for Organisation Strategy i.e.

Yes, it can be thought of as an automated test to check the document link is present, followed by a manual check of the document itself.

If the document fails the manual check (IATI sampling), the indicator cannot be scored as an IATI publication. Points might be awarded if the document can be consistently found via the manual survey. (Details on the overall methodology which will remain unchanged can be found in the technical paper).

AndieVaughn commented 7 years ago

We recommend that organization file documents for CY and CY-1 are accepted for organizational file documents.

There is a possibility that donors could score differently each year just based on the timing of the Index and when specific reports are published. That penalty wouldn’t make sense. For example, if the cut-off for published data is December 31st and updated strategies come out in the spring, the reviewed strategy would be for the previous year and not the current year, even though another one would be in development and just short of publication. Given fiscal years vary, a grace period of an additional 12 months makes sense when performing manual checks.

johnadamsDFID commented 7 years ago

I agree with AndieVaughn - this is even more important for Annual Reports which are normally scheduled for several months after fiscal year close.

carlelmstam commented 7 years ago

I agree with Andre and John above. Carl, Sida

andylolz commented 7 years ago

Thanks for the comments here. This test will remain as proposed.

On the question of a grace period for organisation documents: Annual reports which are up to date within their regular cycle, i.e. the organisation publishes an annual report a year behind, the most recent document within this time frame are accepted. Further details will be provided in the technical paper outlining the new methodology in its entirety.