search

open-contracting / standard

Documentation of the Open Contracting Data Standard (OCDS)
http://standard.open-contracting.org/
Other
139 stars 46 forks source link

Deprecate eligibilityCriteria and add exclusionGrounds #901

Closed jpmckinney closed 2 years ago

jpmckinney commented 5 years ago

Background

In OCDS, selection criteria (for example, the criteria used to qualify bidders in a selective procedure in the sense of the GPA) are often mapped to eligibilityCriteria.

However, selection criteria are not the same as eligibility criteria. For example, the UNDP has qualified but ineligible bidders, e.g. due to suspension or debarment:

… all vendors must be qualified, as well as eligible.

Eligible vendors are qualified vendors that have not been temporarily suspended or debarred by UNDP or another UN agency, fund or programme.

eligibilityCriteria was added in #139, and is described as:

A description of any eligibility criteria for potential suppliers.

At the time, awardCriteria was incorrectly termed selectionCriteria, which was fixed in #162; the confusion might have started in #140.

Proposal

An effective solution depends on how frequently eligibilityCriteria is used to disclose selection criteria (a very common concept in procurement) versus eligibility criteria (rare).

If the field is mainly used for selection criteria, then adding a new selectionCriteria field would create a situation in which there are two terms used to disclose the same concept.

If the field is rarely used, or only used for eligibility criteria, then we can add a new selectionCriteria field, and clearly explain the difference between the two.

We should also avoid the terms 'eligible' and 'eligibility' in all related guidance and extensions, unless that is actually what we mean.

ColinMaudry commented 4 years ago

In a nutshell:

Eligible selected = qualified

Updated according to the next comment.

jpmckinney commented 4 years ago

I think size tends to fall into selection criteria.

For financial health, it depends. If it's something like "declared bankruptcy", it's an exclusion ground / eligibility criterion. If it's something like "minimum $1M revenue" (as a proxy for long-term stability), it's a selection criterion.

ColinMaudry commented 4 years ago

If I get it right, the purpose of this issue is to:

  1. scan the docs and schema for misuses of "eligibility" and "selection"
  2. assess the use of eligibilityCriteria for selection criteria

More generally, in 1.2 we must pave the way towards straight definitions and consistent structures (xxxCriteriaDetails + structured array of criteria) for all criteria in 2.0.

I'll start with 1. and we can discuss how to tackle 2. I understood the data team has a copy of all the OCDS data in the universe. We should be able to extract samples per publisher and analyse them.

ColinMaudry commented 4 years ago

In the schema, I believe the description of awardCriteria is problematic. It refers both to award criteria (assessment of the goods/services :heavy_check_mark: ) and selection criteria (assessment of the tendering party :x:):

Any detailed or further information on the award or selection criteria

@jpmckinney Is removing "or selection" a breaking change?

ColinMaudry commented 4 years ago

I found no misuse of "eligibility" in the docs nor in the schema. It's actually mentioned only two times:

jpmckinney commented 4 years ago

I created #1071 for awardCriteriaDetails.

For this issue, let's add detail to the definition, with some examples, e.g. "For example, a supplier might be ineligible if they have been temporarily suspended or debarred by the buyer."

The issue also proposes adding a tender.selectionCriteria field.

To assess the use of eligibilityCriteria, please ask the Helpdesk to query the database to assess how many publishers use it, and to share some example values. I think @pindec might be best to do this.

pindec commented 4 years ago

I've shared Helpdesk's previous research on the topic with @ColinMaudry and will update with some more recent examples.

ColinMaudry commented 4 years ago

Analysis of the usage of eligibilityCriteria by @pindec: https://docs.google.com/spreadsheets/d/1WgXKTfaHQKrIhMdUMQuK_TAPx19T8DXhaLKTykSoeL0/edit?ts=5f71ec09#gid=386792540

My understanding is that eligibilityCriteria is somewhat used, but not much.

The other 11 are OK.

Conclusion: the publishers that use eligibility generally use it correctly.

Based on this analysis, in the scope of the 1.2 update, I suggest:

In 2.0 we I guess we would rename these text fields xxxDetails and add array fields.

jpmckinney commented 4 years ago

@ColinMaudry Sounds good! I suggest working on each of the two fields in separate PRs, to make them easier to merge.

JachymHercher commented 3 years ago

An effective solution depends on how frequently eligibilityCriteria is used to disclose selection criteria (a very common concept in procurement) versus eligibility criteria (rare). [...] it's an exclusion ground / eligibility criterion. [...]

Three comments:

  1. Assuming from the discussion above that eligibility criteria would in the EU be the same as the exclusion grounds, then those are very common - every procedure must have them. However, since they were (accidentally) not part of the TED 2.0.9 forms, data about them often wasn't published (and is instead stored only in the ESPD).

  2. Within this issue, shall we also define eligibilityCriteria and selectionCriteria? I think we should, as "eligibility" is not clear enough on its own - it doesn't appear in the GPA, EU law, nor UNCITRAL model law.

    As far as I remember, the conceptual difference between "exclusion grounds" and "selection criteria" is that selection criteria are linked to the supplier's ability to perform the contract (turnover, experience, etc.), while exclusion grounds are not (e.g. money laundering, tax evasion, labor standards). I would base the definition on that. @ec-mcs could probably remind us whether that is correct.

    (There are also two "corollaries" stemming from the above:

    • Exclusion grounds are not set by the buyer, but by a higher administrative power (EU-level and state/regional level). Selection criteria are set per procedure and the buyer has full discretion in setting them.
    • Selection criteria are often used for the selection of a limited number of participants for the second stage, while exclusion grounds are not.

    For both of the corollaries, I think the EU has some exceptions though (i.e. in some cases there are buyer-specific exclusion grounds; and in some cases you can choose a limited number of candidates based on exclusion grounds), but I would have a tendency to treat those as jurisdiction-specific anomalies and not reflect them in OCDS.)

  3. What about "conditions for participation" that, both conceptually and in EU law, don't really fit neatly in the exclusion grounds nor selection criteria category? In eForms, these are in Other Requirements (BG-705) and include "participation reserved for sheltered workshops / organisations pursuing a public service mission", "participation requires security clearance" as well as the, slightly different, "tenderer must disclose names and qualifications of staff".

    My tendency would be to just assume they fit in selectionCriteria and, if necessary, deal with them another day :-). (E.g. once the ESPD team figures out where they belong :-) .)

jpmckinney commented 3 years ago

Another option is, given the poor semantics and infrequent terminology of eligibilityCriteria, we simply deprecate it, and introduce exclusionGrounds and selectionCriteria with proper semantics (the latter is added in #1072).

As for conditions for participation, we similarly have an Other Requirements extension, which can change once it becomes clearer where these fit conceptually.

JachymHercher commented 3 years ago

Sounds good. GPA does mention "may exclude [...] on grounds" as one of the general "conditions for participation". (UNCITRAL talks only about (qualification) "criteria".)

jpmckinney commented 3 years ago

https://www.wto.org/english/docs_e/legal_e/rev-gpr-94_01_e.htm

Where there is supporting evidence, a Party, including its procuring entities, may exclude a supplier on grounds such as: bankruptcy;

  • false declarations;
  • significant or persistent deficiencies in performance of any substantive requirement or obligation under a prior contract or contracts;
  • final judgments in respect of serious crimes or other serious offences;
  • professional misconduct or acts or omissions that adversely reflect on the commercial integrity of the supplier; or
  • failure to pay taxes
jpmckinney commented 3 years ago

Some other language for inspiration: https://www.procurementjourney.scot/route-3/develop-documents/exclusion-selection-and-award-criteria/exclusion-criteria

jpmckinney commented 3 years ago

I'm splitting this issue as I suggested in https://github.com/open-contracting/standard/issues/901#issuecomment-700363876 so that selection criteria are dealt with separately in #1120

This issue is about the fate of eligibilityCriteria: whether to change its description or deprecate it in favor of exclusionGrounds. The comments starting from https://github.com/open-contracting/standard/issues/901#issuecomment-728112974 offer options or what language to use in the description.

Regarding "conditions for participation," the GPA article with that title discusses both selection criteria (e.g. legal and financial capacities, commercial and technical abilities) and exclusion grounds.

UNCITRAL Article 9. Qualifications of suppliers and contractors also has a mix of the two:

(a) That they have the necessary professional, technical and environmental qualifications, professional and technical competence, financial resources, equipment and other physical facilities, managerial capability, reliability, experience and personnel to perform the procurement contract; (b) That they meet ethical and other standards applicable in this State; (c) That they have the legal capacity to enter into the procurement contract; (d) That they are not insolvent, in receivership, bankrupt or being wound up, their affairs are not being administered by a court or a judicial officer, their business activities have not been suspended and they are not the subject of legal proceedings for any of the foregoing; (e) That they have fulfilled their obligations to pay taxes and social security contributions in this State;

Since the two are mixed in these documents, there's a possibility that some jurisdictions also mix them. The EU separates them into selection criteria and exclusion grounds, but I don't know that this distinction exists everywhere.

As such, we might need a common model, e.g. an array of participationConditions or participationCriteria, where there's the option to classify/categorize the criterion in order to preserve the distinction where one exists.

So, we should first decide on that last question, and if we chose to have separate fields, then we can continue with the above approach before the horizontal line.

cc @ColinMaudry

JachymHercher commented 3 years ago

Since the two are mixed in these documents, there's a possibility that some jurisdictions also mix them. The EU separates them into selection criteria and exclusion grounds, but I don't know that this distinction exists everywhere.

I understand the cautionary principle, but I think it would be good to find at least one example of a jurisdiction that mixes them, just to be sure we are not being unnecessarily flexible.

As such, we might need a common model, e.g. an array of participationConditions or participationCriteria, where there's the option to classify/categorize the criterion in order to preserve the distinction where one exists.

If we go into modeling individual criteria (or into anything more sophisticated than one or two text fields for entire families of criteria), I think we should be as close as possible to the European Single Procurement Document (ESPD). Main reasons, as far as I know:

The basic structure of the ESPD (looking e.g. at criteria section of the ESPD documentation) is having individual instances of TenderingCriterion which then have property groups and properties. The property "CriterionTypeCode" works as a classification and distinguishes exclusion and selection "criteria" (e.g. 'criterion.exclusion.something', 'criterion.selection.something').

jpmckinney commented 3 years ago

I understand the cautionary principle, but I think it would be good to find at least one example of a jurisdiction that mixes them, just to be sure we are not being unnecessarily flexible.

Good point. @duncandewhurst @yolile Have you come across any governments whose source systems mix selection criteria (professional, technical, financial, etc. qualifications) and exclusion grounds (e.g. being bankrupt or suspended) in the same data element/structure?

I think we should be as close as possible to the European Single Procurement Document (ESPD).

Indeed – I have a feeling we'll need to do an OCDS extension for ESPD. For this issue, we can focus on "whether to change eligibilityCriteria's description or deprecate it in favor of exclusionGrounds" (assuming we find no/rare systems that mix selection criteria and exclusion grounds).

ColinMaudry commented 3 years ago

Exclusion grounds in eForms: https://docs.google.com/spreadsheets/d/1Nh_HjyYuNk8wuu_Tb0kcPDyV6j8pAMgXqJ3MgwpXZjQ/edit#gid=202916689&range=121:121

ID Name Description
BG-701 Exclusion Grounds The brief description of criteria regarding the personal situation of tenderers that may lead to their exclusion. This must include a list of all such criteria and indicate required information (e.g. self-declaration, documentation). This can also include specific national exclusion grounds.
ColinMaudry commented 3 years ago

Deprecating eligibilityCriteria in favor of exclusionGrounds sounds like a good idea.

Looking at the bigger picture, we must set a design pattern for cases where we have core unstructured data that may be structured with an extension. For instance, we work on a core selectionCriteria field #1120, while an extension enables the publication of structured data: https://github.com/open-contracting-extensions/ocds_selectionCriteria_extension.

There is a pattern that has worked pretty well, in my opinion:

Would that work for exclusionGrounds? One field could be core while the other would be in an extension.

jpmckinney commented 3 years ago

The linked extension has a different pattern, which I think is preferred: https://extensions.open-contracting.org/en/extensions/selectionCriteria/master/schema/

{
  "tender": {
    "selectionCriteria": {
      "description": "unstructured data",
      "criteria": [
        // structured data
      ]
    }
  }
}

We can do something similar for exclusion grounds.

ColinMaudry commented 3 years ago

@duncandewhurst @yolile Have you come across any governments whose source systems mix selection criteria (professional, technical, financial, etc. qualifications) and exclusion grounds (e.g. being bankrupt or suspended) in the same data element/structure?

No feedback from @duncandewhurst and @yolile. I assume mixed participation conditions didn't catch their attention.

Shall we (I) proceed with tender.exclusionGrounds and tender.selectionCriteria (#1120)?

yolile commented 3 years ago

@duncandewhurst @yolile Have you come across any governments whose source systems mix selection criteria (professional, technical, financial, etc. qualifications) and exclusion grounds (e.g. being bankrupt or suspended) in the same data element/structure?

@jpmckinney @ColinMaudry no that I'm aware of!

duncandewhurst commented 3 years ago

@duncandewhurst @yolile Have you come across any governments whose source systems mix selection criteria (professional, technical, financial, etc. qualifications) and exclusion grounds (e.g. being bankrupt or suspended) in the same data element/structure?

@jpmckinney @ColinMaudry no that I'm aware of!

Likewise!

JachymHercher commented 3 years ago

When resolving this issue, besides changing/removing 'eligbilityCriteria' in documentType, we should also either expand the definition of evaluationReports to also cover eligbility/exclusion/selection criteria, or we should add a eligibility/exclusion/selectionReports code.

ColinMaudry commented 3 years ago

Regarding document types, I think we should:

ColinMaudry commented 3 years ago

@jpmckinney @JachymHercher Here is my proposal for exclusionGrounds:

The criteria regarding the situation of a tenderer that can lead to its exclusion from the process. Typical exclusion grounds include bankruptcy criminal convictions, presence on a blacklist or failure to pay taxes.

I have reused wording from eForms and GPA and made it more fluid.

It doesn't mention the fact these exclusion grounds are usually defined by a "higher administrative power" than the buyer, I thought it might be more confusing than helpful.

ColinMaudry commented 3 years ago

In https://www.procurementjourney.scot/route-3/develop-documents/exclusion-selection-and-award-criteria/exclusion-criteria, bankruptcy is in the "may ask" column, thus less typical than "criminal convictions". I'll update my proposal.

ColinMaudry commented 3 years ago

Closed with #1296

JachymHercher commented 3 years ago

Regarding document types, I think we should:

  • deprecate "eligibilityCriteria" in favor of "exclusionGrounds", mirroring the fate of Tender.eligibilityCriteria
  • after assessing its actual usage, possibly updating the description of "evaluationReports" to explicitly mention the exclusion and selection phases (+ the evaluation of the bids?).
  • create a "selectionCriteria" document type

@ColinMaudry I don't see this being resolved as part of the PR closing this issue, has this been done somewhere else? Just to make sure it doesn't fall through the cracks.

ColinMaudry commented 3 years ago

@JachymHercher Right, I forgot to take care of the document types! Thanks!

JachymHercher commented 2 years ago

The PR is ready.

1) The following seems straightforward: 

> deprecate "eligibilityCriteria" in favor of "exclusionGrounds", mirroring the fate of Tender.eligibilityCriteria
> create a "selectionCriteria" document type

In both cases I have reused the wording used in `exclusionGrounds` and `selectionCriteria`. 

(Note: `selectionCriteria` speaks about "tenderers", but should probably be about "potential tenderers"?)

2) The description of selectionCriteria is 

> The minimum requirements for tenderers to participate in the contracting process. Selection criteria ensure that a tenderer has the legal and financial capacities and the technical and professional abilities to perform the contract to be awarded. More structured information can be provided using the selection criteria extension.

while we also have codes for 

'economicSelectionCriteria' 

> Documentation that describes the economic selection criteria.

and 'technicalSelectionCriteria'. 
> Documentation that describes the technical selection criteria.

I am a bit surprised that someone would have documents discussing these separately. Are these fields used? If yes, I would suggest minor rewording to bring them in line with the description of `selectionCriteria`. 

> Documentation that describes the minimum economic, financial and legal requirements for potential tenderers to participate in the contracting process.

> Documentation that describes the minimum technical and professional requirements for potential tenderers to participate in the contracting process.

(Alternatively, we could break this into more (e.g. four) subtypes of documents, but I would do that only once there is specific user demand.)

3) > after assessing its actual usage, possibly updating the description of "evaluationReports" to explicitly mention the exclusion and selection phases (+ the evaluation of the bids?).

This could be more complicated, so I've opened up a new issue in https://github.com/open-contracting/standard/issues/1463.
jpmckinney commented 2 years ago

I am a bit surprised that someone would have documents discussing these separately. Are these fields used? If yes, I would suggest minor rewording to bring them in line with the description of selectionCriteria.

You might be right. This is based on the EU standard forms having checkboxes for "Selection criteria as stated in the procurement documents" and "Selection criteria as stated in the procurement documents". https://standard.open-contracting.org/profiles/eu/latest/en/forms/F02/#section-iii

I imagine most buyers just have one big PDF, but I guess in principle that PDF can be broken up into separate documents.

We could omit them, but there is no downside to their inclusion. We can wait for peer review to accept/reject them.

JachymHercher commented 2 years ago

If these checkboxes are the only reason to include them, then I would drop them. They were added to the forms at a later stage than the other fields and sections around them and their point was just to allow the buyer to declare "I have fulfilled my legal responsibility". I don't think it was motivated by actual procurement documents being split this way. (We can wait for peer review to reject them, but we could also just wait for someone to propose it in case they have the need :). )

jpmckinney commented 2 years ago

Okay, we can remove them in the same PR and create an issue on https://github.com/open-contracting-extensions/european-union/issues to update the guidance there.

JachymHercher commented 2 years ago

Done in https://github.com/open-contracting/standard/commit/41fb2cdf59fffa136f3d5e48e8b948933defa607 and https://github.com/open-contracting-extensions/european-union/issues/95.

I'm not 100% sure I've marked this correctly in the change-log.