Check schema field and codelist code descriptions against OCDS, ePO, UNCITRAL glossaries

[jpmckinney]

Same as #826 but for OCDS 1.2 due to versioning policy.

• The EU e-Procurement Ontology working group has a list of glossary terms that link to this spreadsheet. I haven't checked if the ontology itself re-uses the glossary definitions verbatim.
• UNCITRAL glossary
• EBRD glossary
• eForms
• OCDS glossary

[jpmckinney]

Copying comment from https://github.com/open-contracting/standard/issues/380#issuecomment-519951704:

UNCITRAL has a term matching the semantics of 'direct':

Single-source procurement: A method of procurement of last resort which main distinct feature is the absence of competition since the invitation to present a quotation or proposal is addressed only to one supplier or contractor.

[jpmckinney]

http://www.mapsinitiative.org/methodology/MAPS-glossary.pdf

[ColinMaudry]

We could work toward having a taxonomy, connecting concepts according to various properties (equivalent to, similar to, related, broader, narrower), using the SKOS ontology (W3C recommendation) and publishing it using Skosmos(website, Github, example of taxonomy).

[ColinMaudry]

Working document: https://docs.google.com/document/d/1lXRWIOjHjhK5KvVrfuC1MBAMl1JvoV9zBCR7Lt5UcUM/edit

[ColinMaudry]

@jpmckinney Are the code lists available in tabular format?

[jpmckinney]

@ColinMaudry I can prepare that. Do you want one sheet with all codelists, or one sheet per codelist?

[ColinMaudry]

On sheet with all. Thanks!

[ColinMaudry]

Four columns: codelist, code, code label, code description.

[jpmckinney]

From the schema/codelists directory, in fish shell, I ran:

for i in *.csv; if test $i != 'currency.csv'; csvcut -c Code,Title,Description $i | csvstack -g (string replace .csv "" $i); end; end | sort > ../codelists.csv

Uploading the CSV as TXT: codelists.txt

[ColinMaudry]

You don't use xsv yet ?! All the cool kids do :nerd_face:

[jpmckinney]

From #902: We might find other places where we want to replace "contract" with "contract, framework agreement or dynamic purchasing system".

[JachymHercher]

From #902: We might find other places where we want to replace "contract" with "contract, framework agreement or dynamic purchasing system".

Is this a question of precision and/or "apparent audience"/readability? If just the former, then we might not need to change anything, assuming that https://github.com/open-contracting/standard/issues/896 results in contracts covering also frameworks and dynamic purchasing systems. If also the latter, then essentially repeating parts of other definitions is something we should have a policy on (probably part of https://github.com/open-contracting/standard/issues/850). My reflex would be not to do it, because it makes things longer and makes updates hard / impossible compared to a single source of truth approach.

[jpmckinney]

Good point. I am happy with an approach that reduces repetition.

[ColinMaudry]

@jpmckinney @JachymHercher The glossary now includes OCDS schema terms and OCDS codes.

I have hidden the OCP terms that were not relevant for the addition in a glossary of procurement, usually because they are specific to OCDS (release, ocid, awardID, etc.).

I have matched the following sources with the remaining terms:

• MAPS glossary
• ePO glossary
• UNCITRAL glossary
• EBRD glossary

For each of these sources, I have tagged the terms as either (see column Relevant) in each source tab:

• "Relevant" (the term would be a good entry in the OCP glossary)
• or "Added" (the term matches an existing entry and has been added as such)
• or nothing (the term isn't relevant for the glossary, usually because it's too specific, or not close enough to the scope of OCDS)

Many terms would be good additions to the glossary. I suggest that you have a look and that we call to discuss the next steps.

[ColinMaudry]

I start adding eForms

[ColinMaudry]

@JachymHercher Do you know where I can find the latest definitions for eForms BTs? I'm not sure I have the latest spreadsheet.

[jpmckinney]

Awesome! Thanks for the update, @ColinMaudry. For eForms, see the annex linked from this page. There is another spreadsheet for codelists, but I can't find it quickly.

For a call, I'll send an email.

[JachymHercher]

The codelists are at https://op.europa.eu/en/web/eu-vocabularies/e-procurement/tables (click codelist and then "browse content").

[ColinMaudry]

I have added the terms from eForms annex and matched them to OCDS fields or codes when relevant.

[ColinMaudry]

I have added comments when an eForm term probably matches a field or code from an OCDS extension.

[jpmckinney]

A mapping between eForms and the eProcurement Ontology is at https://github.com/eprocurementontology/eprocurementontology/blob/v2.0.2/v2.0.2/03-Analysis%20and%20design/eForms2ePO/Ontology_eForms_Mapping_New%20Regulation.xlsx (https://github.com/eprocurementontology/eprocurementontology/issues/277)

I did a bit of tidying in https://docs.google.com/spreadsheets/d/1Pj4mMIPU-UosrA2rR6Ty7mXNLLOV9sfH/edit#gid=1167530975

It looks like 147 out of 284 BT/BGs have an ePO definition (which might not mention the ePO term). 57 have ePO working group comments (40 of which have no ePO definition).

[ColinMaudry]

I have added most of eForms code in a new sheet from the eProcurement code tables.

I have skipped the following codelists:

• Applicability codes (irrelevant)
• CPV and CPVS (too big)
• Change corrig justification (irrelevant)
• Communication justification (irrelevant)
• Country (irrelevant)
• Currency (irrelevant)
• Environmental impact (irrelevant)
• Language (irrelevant)
• NUTS (too big)
• Permission (irrelevant)
• Time period (irrelevant)
• Usage (irrelevant)

[ColinMaudry]

I have matched eForms codes with OCDS schema fields and OCDS codes.

[jpmckinney]

I've updated the glossary:

• Removed the "To do" tasks:
  • Add eForms terms
  • Check if ePO has eForms mapping https://github.com/eprocurementontology/eprocurementontology/issues/257
  • Extend the glossary with relevant terms
• Cleared the "Notes" columns where updates have been made
• MAPS, UNCITRAL, EBRD, GPA are reviewed

I won't review ePO and eForms as they are too long, so I'll trust that they are mapped correctly.

I consider the crosswalk complete. Once we've worked on the other semantic issues, we can review the remaining concepts to see if any definitions can be improved based on the crosswalk.

[ColinMaudry]

Since many OCDS definitions have moved in the 1.2-dev branch, we should also update them in the glossary.

[jpmckinney]

The glossary has been very useful in clarifying the main concepts. For the remaining, less "core" concepts, there is relatively low overlap across the different sources, and in some cases the source has a much narrower definition.

I think keeping the OCDS definitions up-to-date will be too tedious, but it is something we can do if/when necessary. After all, the main value of the crosswalk is to find the other sources' definitions, not OCDS'.

I have linked to the crosswalk from the schema style guide https://ocds-standard-development-handbook.readthedocs.io/en/latest/meta/schema_style_guide.html#field-and-code-descriptions

I think we covered all the key terms that are likely to change based on other sources' definitions (buyer, procuring entity, tender, award, contract, amendment, etc.).

If we notice a lack of clarity, we can open an issue about an individual field, and reference the crosswalk.