open-contracting / standard

Documentation of the Open Contracting Data Standard (OCDS)
http://standard.open-contracting.org/
Other
138 stars 46 forks source link

Organization identifier guidance: Look into ISO 6523 #987

Closed jpmckinney closed 1 year ago

jpmckinney commented 4 years ago

Used by European Commission and presumably other governments. Identifier schemes can be added for a small fee. It's not as open as org-id.guide, but we should support and document its use, given that governments are using it.

pindec commented 3 years ago

The list of ‘organization identification schemes’ held by this standard is around 200 items long. Each organisation identification scheme is issued an ICD (International Code Designator) value.

Actual registration bodies for organisations are a minor subset of ICDs, so not all items are suitable for use in OCDS to uniquely identify organizations involved in contracting processes. Many schemes provide internal-only identifiers, not all identifiers relate to organizations, and not all organization identifiers are related to legal registration.

This may be due to the initial use case around the adoption of a European system of e-invoicing, and by the needs of the various organisations who have their own internal identification scheme, e.g. "0187 Amazon Unique Identification Scheme" (To provide identifiers for properties, classes, groups, or lists of data and objects specified by or used by Amazon.com, Inc. and its Affiliates), "0163 US-EPA Facility Identifier" (Facilities regulated, monitored, or of environmental interest by the United States Environmental Protection Agency).

Of the suitable items, some are already in org-id e.g. 0199 Legal Entity Identifier (LEI), XI-LEI.

This implies we should only encourage governments to use ISO 6523 if we also specified which sub-set of items were suitable to use to publish organization identifiers in OCDS (i.e. any non-internal registries that relate to the legal basis of the given organisation). Alternatively (and more straightforwardly), since that requires research, plus a cross-check with org-id, we should use that research time instead to ensure all suitable schemes are in org-id.

Also noting that a canonical version of the ICD list is not easy to locate: the PDF version has a final ICD value of 0189, and the host website of that PDF doesn't mention Farance Inc. as the current Registration Authority. The spreadsheet version held by a European Commission body that just contains the names of the organisation identification schemes seems to be more up-to-date (highest ICD value 0209).

@jpmckinney, do you have any additional use cases and/or example data to hand please?

jpmckinney commented 3 years ago

The eForms Policy Implementation Handbook has an Organization Identifier section:

The organisation identifier is a crucial field. It is necessary in order to answer questions such as: "How much money has a buyer spent in a year" "How often do procedures run by a given buyer receive only a single bid" “Does a certain buyer always buy from a certain supplier", "Does a certain supplier never/always bid together with another specific supplier", etc.

Answers to these questions are important for managers of national procurement systems, as well as for companies doing business planning, NGOs, academics, etc.

eForms accept the same list of identifier schemes as eInvoicing, which is based on the international code designator (ICD) falling under ISO 6523. An identifier scheme can be added to the ICD for a small fee.

However, not all identifiers are created equal. Good identifier schemes track mergers and acquisitions, add additional information about a company such as its turnover or headcount, are available in searchable open data formats, etc. An example of a high-quality identifier scheme is the Legal Entity Identifier, which is available in all EU countries.

The tailoring decision for the organisation identifier consists of deciding whether only certain ICD identifier issuers will be accepted (this will then be implemented by additional business rules, see below) and ensuring that any identifier scheme to be used is part of the ICD codelist.

Since source data from EU Member States will follow that guidance, we should at least offer a straightforward way of mapping it to OCDS.

I don't know if ISO 6523 is used in current TED or if it's a new requirement for eForms. It's used in eInvoicing, but I don't know if that data is open/public.

kd-ods commented 3 years ago

Since source data from EU Member States will follow that guidance, we should at least offer a straightforward way of mapping it to OCDS.

This is something we've been considering with respect to org-id and org-identifiers in general: the necessity of a cross-walk between identifier scheme registers.

It's interesting that the eForms guidance takes the iso list as a starting point but is candid about the lack of suitability of many of the schemes on it:

The tailoring decision for the organisation identifier consists of deciding whether only certain ICD identifier issuers will be accepted.

kd-ods commented 3 years ago

I’ve now looked through the 204 ‘organization identification schemes’ held by the iso 6523 standard.

The outcome is that of those 204 identifier schemes, 13 are in org-id and there are roughly another 8 that should/could be added to org-id. The other 183 cannot really be considered good quality identifiers IMO - the schemes are for very specific purposes, either internal to an organisation or as part of a collective exercise (e.g. telecommunications and network addressing). They are not schemes that relate to the legal recognition/establishment of an organisation.

These are the next steps that I’d recommended:

jpmckinney commented 3 years ago

Thanks, @kd-ods! Could you list the 13 schemes that are already in ORG-ID and the 8 schemes to add (as well as their correspondence in ISO)?

For the other 183, did you track why a given scheme wasn't good quality, or did you just take note of which were good? If the former, it'd be useful to share in case we get a request to add one of those at a later date.

kd-ods commented 3 years ago

@jpmckinney - I'm not sure how well the table below will format, but here's where it's from: my analysis sheet. For the other 183 lists, it became apparent quickly that most of them were what we'd call 'local' lists in org-id parlance: 'Maintained by a single organisation for their own business purposes'.

iso 6523 org identifier schemes already listed in org-id:      
       
Code Identifier scheme name org-id link Notes
0002 System Information et Repertoire des Entreprise et des Etablissements: SIRENE http://org-id.guide/list/FR-RCS  
0007 Organisationsnummer http://org-id.guide/list/SE-ON  
0009 SIRET-CODE http://org-id.guide/list/FR-RCS  
0093 Revenue Canada Business Number Registration (EDIRA compliant) http://org-id.guide/list/CA-CRA_ACR  
0151 Australian Business Number (ABN) Scheme http://org-id.guide/list/AU-ABN  
0154 Identification number of economic subjects: (ICO) http://org-id.guide/list/CZ-ICO  
0169 Swiss Federal Business Identification Number. Central Business names Index (zefix) Identification Number   This ID has been superseded by that in 0183 (according to the org-id entry for Switzerland). See org-id link there.
0183 Numéro d'identification suisse des enterprises (IDE), Swiss Unique Business Identification Number (UIDB) http://org-id.guide/list/CH-FDJP I think this means 0169 is no longer relevant
0188 Corporate Number of The Social Security and Tax Number System http://org-id.guide/list/JP-JCN  
0191 Company Code (Estonia) http://org-id.guide/list/EE-RIK  
0192 Organisasjonsnummer http://org-id.guide/list/NO-BRC  
0199 Legal Entity Identifier (LEI) http://org-id.guide/list/XI-LEI  
0200 Legal entity code (Lithuania) http://org-id.guide/list/LT-RC I think this is a match
0208 Numero d'entreprise / ondernemingsnummer / Unternehmensnummer http://org-id.guide/list/BE-BCE_KBO This appears to be a match
       
       
iso 6523 org identifier schemes that should be considered for org-id:      
       
Code Identifier scheme name List type Notes
0029 The All-Union Classifier of Enterprises and Organisations Primary This ICD was registered in 1991 by the USSR. So probably doesn't exist in this form now?
0037 LY-tunnus Secondary ICD registered by the National Board of Taxes, Finland, 1992. Covers: "All organizations in Finland included juridical persons and associations"
0060 Data Universal Numbering System (D-U-N-S Number) Third-party org-id has an open issue about whether to add DUNS as a list: https://github.com/org-id/register/issues/318
0158 Identification number of economic subject (ICO) Act on State Statistics of 29 November 2'001, § 27 Secondary This Slovakian identifier should be added to org-id. It's similar to the Czech id
0189 European Business Identifier (EBID) Third-party  
0196 Icelandic identifier - Íslensk kennitala Primary  
0201 Codice Univoco Unità Organizzativa iPA Local Not much detail to go on, but possibly could be useful to have in org-id depending on level of org unit
0206 Registre du Commerce et de l’Industrie : RCI Primary https://www.rci.gouv.mc/rc/
kd-ods commented 3 years ago

This work on adding and updating iso6523-compliant org-id lists will be picked up over the next weeks. Issues in the org-id repo relating to this work will be tagged iso 6523.

kd-ods commented 3 years ago

@duncandewhurst - I just noticed that the list of ISO 6523 ICD values held by the BIS e-invoicing standard now goes up to 213 (i.e. there are 4 more codes than when I first looked at this last year).

So:

1) I'll need to take a look at those 4 lists and see if they are on org-id or need to be added; and

2) We need to think about this element of aligning with ISO 6523:

The challenge here will be to keep it up to date with any new, quality id schemes being added to iso 6523 since the list isn’t online. Making contact with the Registration Authority for the standard, Farance Inc. and requesting notifications is one option. The other is just leaving it to publishers to alert OCDS if a new ICD should be added to the table.

duncandewhurst commented 3 years ago

Sounds good! Another option for maintaining alignment is to add a step to the deploy process to prompt the deployer to check for new codes. @jpmckinney do you have a preference?

jpmckinney commented 3 years ago

Doing both sounds best. I’m subscribed to ISO currency announcements, for example.

duncandewhurst commented 3 years ago

I've emailed Farance at the address listed on https://www.iso.org/organization/270240.html

kd-ods commented 3 years ago

Just noting here that I'm going to keep this research list updated until we decide otherwise. My tasks for the moment are:

This will mean that we can use this summary in the workbook to produce the ICD <-> org-id mapping for the OCDS docs.

duncandewhurst commented 2 years ago

I've completed the actions in the previous comment. There are two lists remaining to add.

Edit: I also checked the PEPPOL codelist, there are no further new lists since https://github.com/open-contracting/standard/issues/987#issuecomment-880593771

I've emailed Farance at the address listed on https://www.iso.org/organization/270240.html

Farance never replied so I've sent another email to the info@ address listed on http://farance.com/.

Another option for maintaining alignment is to add a step to the deploy process to prompt the deployer to check for new codes

I've added update instructions in https://github.com/open-contracting/standard-development-handbook/pull/270

JachymHercher commented 2 years ago

@duncandewhurst Out of curiosity, has Farance replied? Also, how did you add the two lists if he did not reply?

duncandewhurst commented 2 years ago

No reply from Farance, as yet. We used the PEPPOL codelist to identify the new lists, which still need to be added to org-id.

odscrachel commented 2 years ago

A note that we had a query today (via the OCDS helpdesk) from a private company trying to find GS1 in org-id https://www.gs1us.org/tools/gs1-company-database-gepir

jpmckinney commented 2 years ago

In https://docs.google.com/spreadsheets/d/1qPPAOx9w4NIfrll66a0Qd2SMizoxGNRUMU6YV2rLuEE/edit#gid=1900375069 the only entry for GS1 refers to GTIN. I'm not sure that GEPIR is listed in ISO 6523?

duncandewhurst commented 1 year ago

Do you mean code 0160?

The description of code 0209 seems to be written to cover all the different types of identifiers issued by GS1:

Code Identifier scheme name Issuer Notes
0209 GS1 identification keys GS1, a global organization. GS1 identification keys and key qualifiers may be used by an information system to refer unambiguously to an entity such as a trade item, logistics unit, physical location, document, or service relationship.

The query in CRM-8346 was specifically about Global Location Numbers which I think is what 'physical location' refers to above.

The structure of the different types of GS1 identifier are described here: https://www.gs1au.org/resources/standards-and-guidelines/identification-numbers/types-of-gs1-id-keys

jpmckinney commented 1 year ago

Hmm, that page doesn't mention GEPIR, which is what @odscrachel had linked to. I'm still not sure whether GEPIR is represented in ISO 6523. The others do seem to be covered by 0209.

As for CRM-8346, I'm not sure that GLNs are appropriate to use as organization identifiers.

duncandewhurst commented 1 year ago

Hmm, that page doesn't mention GEPIR, which is what @odscrachel had linked to. I'm still not sure whether GEPIR is represented in ISO 6523. The others do seem to be covered by 0209.

My reading of it is that GEPIR is a database of the identifiers issued by GS1, rather than a separate set of identifiers.

As for CRM-8346, I'm not sure that GLNs are appropriate to use as organization identifiers.

I think that we'd need to explore it further with the person who sent in the request. According to the GS1 website:

What can GLNs identify?

Legal entities

Any legal or operational entity, such as a company, its subsidiaries, buyer, seller or operating division.

Looking at the page I linked in my previous update, the GLN (and most other GS1 identifiers) includes a 'GS1 company prefix', which seems like an organization identifier:

image

However, searching for 'Microsoft' in GEPIR returns 8 different GS1 Company Prefixes, all with the same 'Entity GLN'. Here's the first 3:

image

duncandewhurst commented 1 year ago

@jpmckinney is the remaining task here just to add the final identifier scheme referenced in https://github.com/open-contracting/standard/issues/987#issuecomment-1101897989 to org-id:

jpmckinney commented 1 year ago

To recap, the issue was labelled as guidance and started with:

[ISO 6523 is] used by European Commission and presumably other governments. Identifier schemes can be added for a small fee. [...] we should support and document its use, given that governments are using it.

And later:

Since source data from EU Member States will follow [the eForms Policy Implementation Handbook Organization Identifier section], we should at least offer a straightforward way of mapping it to OCDS.

However, the ISO 6523 list is not open and we instead check PEPPOL (no idea how frequently it is updated).

We also learned that the ISO 6523 list is mostly (like 90%) not relevant to organization identifiers, as tracked in this sheet.

Among the relevant codes, they have an obvious mapping to OCDS (where a corresponding org-id.guide code exists), and so I don't think any new guidance is needed. As such, this issue's original intention is satisfied.

The issue later covered adding new codes to org-id.guide (all but one done) and discussing whether GS1 codes are covered and what its GEPIR product is.

Anyhow, with one code left (and tracked in org-id's issue tracker), I think we can close this issue.