BUGFIX: Re-add withdrawn codes

[andylolz]

These codes were removed as part of #283, because codelist values were taken directly from DAC XML. However, this DAC XML is missing a number of withdrawn codes, so these should be re-added.

As per the codelist management page, Withdrawn codes are never removed from replicated codelists. So this should be addressed as a bugfix.

[andylolz]

Note that this bug affects any software doing codelist validation. This includes the UK Aid Review.

[markbrough]

I also know of at least one country-level system that periodically attempts to re-sync with these codelists, so would be useful for the withdrawn codes to be included and complete

[andylolz]

I was confused why I’d broken this in my PR back in January. But I checked back, and in my defence I did actually specify there that I hadn’t checked for removed codes:

Here’s what I haven’t done: […]

• Gone through and checked for removed codes (anything removed is a problem, since it should instead be marked as status="withdrawn")

In fact, I did also flag that these withdrawn codes had been removed. It’s very possible that I misunderstood what was decided on this (i.e. whether this should be fixed in the source DAC XML, or just in the replicated XML.) In any case, I definitely dropped the ball in not re-flagging the issue before it was merged.

[PetyaKangalova]

@andylolz and @markbrough thanks for flagging this again. I just got in touch Valerie from OECD to ask what process they follow for removed codes. Next steps: 1) I'll will to hear back from Valerie to see if the OECD DAC keep record of the removed codes. 2) If there isn't a record of any removed codes from the DAC, then I will review this pull request and re-add the removed codes and mark as withdrawn (thanks @andylolz for re-adding the codes in this pull request!).

Hopefully this can be resolved by end of next week and then we can move into reviewing the new updates.

[andylolz]

That’s great. Many thanks @PetyaKangalova.

[markbrough]

Excellent, thanks for the update @PetyaKangalova

[PetyaKangalova]

Brief update - I have not heard back from the DAC as their contact is on leave until 28th of August. I will wait to hear until end of next week (30th of August). If not, I will go ahead with the manual update.

[PetyaKangalova]

@samuele-mattiuzzo and I just received this response from the DAC:

For the time being we do not have longer these removed codes in our database so it’s not possible for us to provide this information in the xml file. We are tracking changes in purposecodes from 2016 only (new codes starting in 2016 and code removed in 2015).

This confirms that we will have to proceed with manually updating the removed codelist before 2016. With focus on preparation for the Members' Assembly, datastore and validator work this month, we will aim to work on this issue towards the end of September/beginning of October.

[andylolz]

Okay, good to hear that this is moving forward. Many thanks for the updates, @PetyaKangalova.

[andylolz]

Great – many thanks for reviewing and merging.

[andylolz]

Is this fix due to be deployed? Is it possible to provide a timeframe?

Thanks

[samuele-mattiuzzo]

@andylolz yes, we'll deploy it possibly early next week. It's just me from the dev team in the office for this week, so there won't be any deployment unless critical

[andylolz]

Great – thanks

[samuele-mattiuzzo]

Deployed http://reference.iatistandard.org/203/codelists/Sector/

[andylolz]

Excellent – thanks