Make a compiledRelease schema

[kindly]

As mentioned in #284 a compiledRelease in an OCDS record has the same schema as a standard OCDS release. However, some of the fields, most notably the release ID and date, make no sense once compiled and they are also mandatory. This means it is not possible to make a compiled release that makes sense and validates.

The best approach would be to make a new sightly modified version of the OCDS release to take this into account. Naming possibly merged-release-scema.json

[duncandewhurst]

It's also worth noting that whilst the merging rules guidance states that the id and date fields should be omitted from a compiled release, the example data on the same page of the docs shows a compiled release with both of these fields included.

We should fix this as when we update the docs as part of the upgrade

[Bjwebb]

@kindly Your current code drops tag from the versioned release. I'm not quite clear that this is the correct behaviour.

[Bjwebb]

Just discussed with @kindly why dropping tag does make sense, because every versioned block has the associated releaseTag.

[jpmckinney]

The issues that @kindly and @duncandewhurst reported haven't been fixed.

[jpmckinney]

Starting to document differences between release-schema.json and proposed compiled-release-schema.json:

• id and date should be omitted from the schema (backwards-compatible)
• tag should be ["compiled"], or it should be omitted from the schema, and 'compiled' should be removed from the releaseTag.csv codelist. I prefer the latter. Making both those changes at once would be backwards-compatible, as only compiled releases should be using the 'compiled' tag.
• No fields should have null values (debatably backwards-incompatible)
• See discussion on identifying (or at minimum dating) compiled releases in #455
• If publisher is added to the release schemas, as in #325, we might want its semantics to be different in the compiled release, i.e. to be the publisher of the compiled release, instead of following the regular merge rules (or, we might want to use a different field name for this).

[JachymHercher]

tag should be ["compiled"], or it should be omitted from the schema, and 'compiled' should be removed from the releaseTag.csv codelist. I prefer the latter. Making both those changes at once would be backwards-compatible, as only compiled releases should be using the 'compiled' tag.

Basic question: why do compiled releases currently have releaseTag set only to 'compiled', instead of containing the tags of all the individual releases and, additionally, 'compiled'? Since the field is already there anyway, wouldn't the "adding" approach remove an anomaly and also have the added value of showing users at first sight how far along the contracting process has progressed?

[jpmckinney]

Tags are not cumulative. The semantics of tag are "what is this release doing?" like "this release is amending the contract". If a prior release amended the tender, it would be incorrect to tag a later release about a contract update as ["tenderAmendment", "contractUpdate"].

That said, tag cannot have those same semantics in a compiled release (the compiled release is about many releases, not this release), and so we could introduce new semantics and accumulate the tags into an array (as you suggest) – instead of the options described previously (requiring that it be ["compiled"] or removing the field from the compiled release).

[ErinClark]

Hi @jpmckinney , I am also just trying to work out a solution for this for users who just want to get the compiled release, but also retain information on the history of which phases it is combining. I was going to add a top level field to the compiled release named 'compiled_tags' which would be an array of all previous tags. Does this sound reasonable?

[jpmckinney]

@ErinClark Until this issue is resolved, it makes sense to add an additional field as you suggest.

[duncandewhurst]

so we could introduce new semantics and accumulate the tags into an array (as you suggest) – instead of the options described previously (requiring that it be ["compiled"] or removing the field from the compiled release).

This would be useful for implementations that use the EU profile because it would be possible to check, using only the compiled release, if both a tender and award release had been published. For the UK, which has issues with linking tenders and awards, currently, you need to check individual releases.

[jpmckinney]

Based on https://github.com/open-contracting/standard/issues/1160#issuecomment-970684975, +1 to accumulating tags in compiled releases.

[odscjen]

It's been a few years since the initial list of differences was made in https://github.com/open-contracting/standard/issues/330#issuecomment-445022030, the following is the updated version based on the outcome of previously mentioned issues and further discussion in this and related threads.

• id should be ocid and the maximum date among the individual releases used to create the compiled release, separated by a hyphen: {ocid}-{date}.
• date should be the maximum date among the individual releases used to create the compiled release.
• tag should include all tags from the individual releases and the additional ["compiled”] tag.
• No fields should have null values (debatably backwards-incompatible)
• publisher should be set to the publisher of the compiled release. This may be different from the publisher of the individual releases. But as the definition of the release publisher is “The original publisher of this release”, semantically the field definition doesn’t need to change.

[odscjen]

@jpmckinney I've manually created a file compiled-release-schema.json using the list above and put it in a draft PR for now. I've assumed here that the purpose of having compiled-release-schema.json is to validate a compiled release against, not for tools creating the compiled release.

• descriptions updated for id, tag, date and publisher and "omitWhenMerged" field removed
• removed null as an option in arrays of strings to enforce the "No fields should have null values" element (which I'm aware causes a bunch of tests to fail, possibly adding minItems to all of those arrays is preferable as long as they're not required?).

Is this schema what you had in mind? If it is then I can move onto making it automatable by manage.py as part of the pre-commit step