Update to v1.1

[yolile]

https://github.com/open-contracting/kingfisher/issues/256

closes #77

[coveralls]

Coverage increased (+0.4%) to 98.144% when pulling 12b6d161a1f07ae728a2e0867bf6b0861a4939c6 on update_to_v1.1 into fc56bfeec216b92ab3595f99d886054be8c5c0d0 on master.

[jpmckinney]

If you run flake8 locally, you won't have to wait for Travis :)

[jpmckinney]

I know the except Exception: pass blocks were present in the original scripts, but we shouldn't ignore all exceptions like that. There are also a variety of other code and style issues, repetitions, etc. from Tim's original version.

Can you:

1. Fix the issues, so that the code is closer to the quality of our other code
2. Change it so that it takes packages from standard input, instead of reading a directory
3. Put test fixtures as files in tests/fixtures instead of using very long strings
4. Rename the command to upgrade, and have it take a required argument which should be two OCDS versions as a colon-separated list, e.g. 1.0:1.1. In future, we'd add support for upgrading between other versions.

[yolile]

@jpmckinney I made the requested changes, but the tests are failing due to a pytest version issue

[jpmckinney]

Thanks! I fixed the build. I'll review PR shortly.

[jpmckinney]

@duncandewhurst Do you have an idea why the old code considered two organizations whose only difference was a contact point to be different organizations? I don't see why that should be the case.

---

Note: Versioned releases within a record package are not upgraded.

Note: The upgrade can lead to information loss if the same organization has a different address, contactPoint, or additionalIdentifiers across occurrences. The code warns in such cases.

---

Moved upgrade functions to new file, and:

• Add the version field to packages only
• Upgrade releases within records
• Fix amendment upgrading to keep additional fields and pattern properties
• Fix organization upgrading to create a more reasonable identifier and avoid adding an empty name
• Warn if there is information loss during organization upgrading
• Fix providerOrganization and receiverOrganization upgrading (they are Identifier's in 1.0)
• Don't overwrite fields if they already exist
• Load the JSON as an OrderedDict, instead of coercing it
• Don't upgrade non-OCDS fields, like contracts/suppliers
• Don't remove empty arrays (previously included for Apache Drill)
• Don't return values, as changes are made in-place anyhow
• Have consistent order of parties
• Suffix functions with the relevant versions
• Make versions a required argument for upgrade command
• Add no-op upgrade functions for 1.0:1.0 and 1.1:1.1
• Add docstrings

[duncandewhurst]

Do you have an idea why the old code considered two organizations whose only difference was a contact point to be different organizations? I don't see why that should be the case.

My guess is that this is due to the guidance in organization section of the release reference, which states:

If a contracting process represents a contract arranged by the department or branch of a larger organization, the legal entity (usually the registered organization) should be described in the identifier section, with details of the branch or department given in the name, address and contact point as relevant.

Presumably to address the case where the procuring entity and the buyer are different departments of the same legal entity.

The guidance on constructing the party id is in line with this, though not reflected in the code (since there isn't a department id field in OCDS):

The ID used for cross-referencing to this party from other sections of the release. This field may be built with the following structure {identifier.scheme}-{identifier.id}(-{department-identifier}).

[jpmckinney]

Thanks, @duncandewhurst ! I've changed the code to also use address and contactPoint to identify the organization, if present.

@yolile Please review and merge if you see no issues.