Pre-qualification for multi-use list

jpmckinney commented 5 years ago

Context

The proposal in this issue is for pre-qualification leading to a multi-use list or similar.

There is another issue for pre-qualification/pre-selection leading to a single request for tender (i.e. the pre-qualification stage produces a single-use list of qualified suppliers): #906

Compared to a single-use list, multi-use lists introduce many more considerations:

Suppliers: Can new suppliers join the list later? If so, at any time, or at specific times? Can suppliers be removed from the list? How long is the list valid?
Competition: Are there competitions between listed suppliers, or is there instead a set of rules to determine which supplier is used in which cases (or both)? Can off-list suppliers participate in competitions by submitting a tender, with or without a request to join the list?
Buyers: Can multiple buyers use the list?

Background

The GPA defines a multi-use list as:

a list of suppliers that a procuring entity has determined satisfy the conditions for participation in that list, and that the procuring entity intends to use more than once

There are examples in Paraguay (linked from this comment: https://github.com/open-contracting/standard/issues/906#issuecomment-519671713) that seem to match this definition of a multi-use list.

The section of the GPA on multi-use lists has more detail on notices and steps, notably:

A procuring entity may maintain a multi-use list of suppliers, provided that a notice inviting interested suppliers to apply for inclusion on the list is … made available continuously

A procuring entity shall allow suppliers to apply at any time for inclusion on a multi-use list

Where a supplier that is not included on a multi-use list submits a request for participation in a procurement based on a multi-use list … a procuring entity shall examine the request

A procuring entity … may use a notice inviting suppliers to apply for inclusion on a multi-use list as a notice of intended procurement

Other procedures that use a list of qualified suppliers multiple times include framework agreements; however, framework agreements might have important differences.

Note that the GPA doesn't mention framework agreements, so multi-use lists might be a super-category that covers framework agreements among others.

Proposal

Given the similarity to framework agreements (the main difference being that a multi-use list doesn't imply a set-up agreement/contract between the government and qualified suppliers), the model for multi-use lists in OCDS should be similar to the one for framework agreements, and in particular open framework agreements (i.e. dynamic purchasing systems).

First stage model

The proposal follows the same considerations for this first stage as in #906.

Second stage model

Like in #906, the qualified submitters (i.e. the multi-use list) can have organization references from a secondStage.candidates array. To model additions to the multi-use list over time, new parties can simply be added with the appropriate role. To model removals from the multi-use list, there's an open issue for removing items from arrays #232; at a minimum, the relevant role can be removed from the party's roles.

A supplier that is not included in the multi-use list would submit a request to participate (thereby modifying the numberOfSubmitters and submitters). As mentioned in #906, there is presently no extension to disclose the requests to participate (i.e. there is no extension that mirrors what the bids extension does for tenders). The decision to include or exclude the supplier from the list is reflected in changes to the party's roles and to the secondStage.candidates array.

Otherwise, the model is fairly similar to #906, except there would be more than one object in the secondStage.invitations array – one for each request for tender using the multi-use list.

jpmckinney commented 5 years ago

@yolile In Paraguay, how long does the list of qualified suppliers last? Can new suppliers apply to join the list at any time, or only at specific times? How frequently is the list updated? Does the government ever cease to recognize a supplier as qualified, or remove a supplier from the list?

jpmckinney commented 5 years ago

@yolile Based on the conversation in #906, I think the OCID for the planning and tender processes in Paraguay can be used as the secondStage.invitations.id. As for where to put the planning information, that's an unresolved issue: #910.

jpmckinney commented 5 years ago

Noting that bids will need to be related to specific invitations.

jpmckinney commented 5 years ago

Here are some questions/use-cases to help assess different models. Preferably, a user should only have to look in one place for information about a particular concept. However, all models require people to look in more than one place to fulfill some use cases; it's therefore a question of which use cases we want to prioritize.

What opportunities are there? (e.g. as a supplier)
What items are being procured?
How does the value of the award relate to the value of the invitation?
Which suppliers participate together?
How many procedures are unsuccessful?
How many times was a procurement attempted before it succeeded?

These questions are answered the same way (using the bids extension) regardless of the modelling decisions above:

Which bidders bid the most?
Which bidders are consistently unsuccessful?

We can add other questions from the Using it, not losing it guide.

jpmckinney commented 5 years ago

Copying relevant comment from #910:

From an 2019-10-07 Skype chat: In Paraguay, there might be multiple planning processes at the same time about (for example) buying trucks. Since they know they will buy trucks this year, they do a prequalification process to get a list of suppliers. These suppliers don't know much about the specific needs (how many trucks, what size, etc.). When the government is ready to tender, the prequalified suppliers are invited to submit a tender.

If that is the case, then I think it makes more sense to have one OCID to get on the list, and then one OCID for each tender that uses the list. The contracting process to get on the list can use regular OCDS fields. However, I'm not sure what value the procurementMethod should have for the tenders, and I'm not sure what relationship code should be used in relatedProcesses.

Otherwise, if we tried to model this all as one process, we'd have multiple relatedProcesses for all the different planning processes, which suggests that model is wrong.

This scenario assumes the government doesn't know in advance whether the PQ list will be used only once; if it did, it could make it part of one process (e.g. like a restricted procedure in the EU), as there is only one planning and one tender.

jpmckinney commented 5 years ago

I'm now looking for other examples of multi-use lists.

In Canada, there are standing offers and supply arrangements (SOSA) (more on standing offers and supply arrangements). In both cases, suppliers are pre-qualified.

A standing offer seems to be like a framework agreement with a single supplier, in which the government does a "call-up" (instead of a "call-off") for good and/or services at pre-arranged prices.
A supply arrangement seems like a framework agreement with multiple suppliers, with mini-competitions. The prices for the range of goods and/or services are negotiable. The procedure might either be competitive bidding or a negotiated procedure.
There might be planning processes relating to each ministry's use of the SOSA, but this information is not captured in a structured way.

As such, SOSA aren't comparable to (my understanding of) pre-qualification in PY.

In Ontario, there are vendors of record (more), which are more flexible than SOSA, in that a ministry can choose to contract with a specific pre-qualified supplier (if the value is under $25,000) or can invite only a subset of the pre-qualified suppliers to a competition (again depending on the value). As such, it breaks the assumption of our FA modelling that all pre-qualified suppliers are invited to all competitions. We can either:

Keep all competitions in the same process, and add a field to list the invited suppliers for each competition (i.e. treat it as an FA).
Have different processes for each competition, and add a link back to the process that established the VOR (i.e. treat it as a generic multi-use list).

Ontario VORs might be more similar to pre-qualification in PY.

jpmckinney commented 5 years ago

From my (limited) experience in the Ontario government, the planning processes for procedures using VORs are independent, and so it would make more sense to treat VORs as generic multi-use lists.

Regarding a question above about how to set procurementMethod, If a ministry contracts with a specific pre-qualified supplier, then it is 'direct'. If it invites only a subset of the pre-qualified suppliers, then it is 'limited'. If it invites all pre-qualified suppliers, then it is 'selective'.

To avoid confusion with other types of pre-qualification (which is just a step that can occur in many different types of procedures), the relationship code in relatedProcesses might be 'multiUseList', to match GPA terminology.

With respect to the use case of a potential supplier trying to find opportunities, we had originally been thinking that a 'selective' procurementMethod should imply (1) the first stage is pre-qualification and (2) any supplier can request to participate. Above, this is not the case; pre-qualification is done as a separate process, and bidding is restricted to the qualified suppliers.

However, I think that this is fine and that the implication is incorrect. A call for tender (that uses a multi-use list of prequalified suppliers) might or might not be an opportunity for a potential supplier, depending on local laws. According to the GPA, an unlisted supplier can still submit. However, in procedures that are not covered by GPA, that is not necessarily the case.

This suggests that we might need a different way to indicate whether new participants can submit to selective procedures – instead of implying that they can submit to any selective procedure.

jpmckinney commented 5 years ago

With respect to re-using the tender block to advertise the opportunity to join a multi-use list:

Across different procurement procedures, suppliers can make many different types of submissions: they can submit a request to join a list of qualified suppliers, to participate in a framework agreement, to submit a tender, to participate in an auction, etc.

Would it be better to put this information in one place (tender, which should perhaps be named something more generic like firstStage or opportunity), to make it easy for potential suppliers to find the different types of opportunities, or is it better to have different places for each?

I think the distinction between first stage and second stage is more important to make clear in the model (and is more relevant to more use cases), than the distinction between different types of first stages (which can be distinguished at the field-level, in the same way that different procedures are distinguished via fields like procurementMethod).

jpmckinney commented 4 years ago

Note: The Law and Economics of Framework Agreements suggests that supply arrangements and standing offers are “more appropriately considered as lists of suppliers than as framework agreements”.

jpmckinney commented 4 years ago

In the case of Paraguay, we are considering one process to create a supplier list (which is not a multi-use list) and then subsequent processes that use the supplier list. The subsequent processes would need to use relatedProcesses to refer back to the process that created the list, in which case we might use a new code like 'supplierList'. Since the result of the first process is to get added to a supplier list, we are considering disclosing that information in an award.

That said, in most cases, the supplier list is used only once, in which case it is perhaps better to simply model this scenario the same as a restricted procedure in the EU – and only add additional processes in the exceptional cases.

jpmckinney commented 4 years ago

JachymHercher commented 3 years ago

Note that the GPA doesn't mention framework agreements, so multi-use lists might be a super-category that covers framework agreements among others.

That is my understanding as well.

In the EU, beyond framework agreements and dynamic purchasing systems, the GPA's multi-use lists are also implemented as "qualification systems" under the sectoral procurement Directive (see Art. 77 of Directive 2014/25/EU). This is the closest to the GPA approach, in the sense that a single qualification system can then lead to many procedures. In practice, I think it is very much comparable to a DPS.

While this instrument is relatively used (~ 1000 notices / year), since it's under the sectoral directive, I think it's a bit off the policymakers' eyes. For eForms, I explicitly asked for feedback on it and received none.

I think something like a qualification system might also exist below-threshold in some EU countries, but I don't have an example.

In practice, I think the same need is increasingly being met by services such as https://amtliches-verzeichnis.ihk.de/, which store structured data on suppliers. Once the service has the supplier's data, the supplier no longer submits his data for qualification, but a buyer, upon received the bid, checks the supplier's data himself. Legally though, this is a standard procedure, nothing special. (ESPD goes in the same direction.)

jpmckinney commented 3 years ago

Is a qualification system like a "supplier list"? e.g. I know many procuring entities that create lists of "authorized suppliers" with whom buyers can contract with much lower procedural overhead.

odscjen commented 2 years ago

adding to this to note that the OCDS pilot data from Canada includes 3 types of standing offers. They're modeling them as an additional field awards/agreementType

JachymHercher commented 2 years ago

Is a qualification system like a "supplier list"? e.g. I know many procuring entities that create lists of "authorized suppliers" with whom buyers can contract with much lower procedural overhead.

Possibly yes, but your description of "supplier list" seems as broad as that of a "multi-use list", i.e. the super-category itself? (E.g. a simple framework agreement with one tenderer would also be a "supplier list"?)

adding to this to note that the OCDS pilot data from Canada includes 3 types of standing offers. They're modeling them as an additional field awards/agreementType

What are they? Are they different to the SOSA and VOR of https://github.com/open-contracting/standard/issues/909#issuecomment-543315075? I'm still naively hoping this call all be somehow squeezed under different types of framework agreements.

odscjen commented 2 years ago

What are they? Are they different to the SOSA and VOR of #909 (comment)? I'm still naively hoping this call all be somehow squeezed under different types of framework agreements.

They are "Regional Master Standing Offer (RMSO)", “National Master Standing Offer (NMSA)”, “Regional Individual Standing Offer (RISO)”. They actually have 5 different types, it's just that only 3 are in the pilot data published. The other 2 are "National Individual Standing Offer (NISO)" and "Departmental Individual Standing Offer (DISO)". So these are sub-types of the SOSA mentioned in #909 (comment) with the subtype being determined by which department/agency and which geographic regions are involved.

jpmckinney commented 2 years ago

@JachymHercher Yes, the standing offers fall within the SOSA classification. Re: whether SOSA and VOR are framework agreements, see: https://github.com/open-contracting/standard/issues/909#issuecomment-554681062 In Ontario, a buyer can do a limited procedure, inviting only a fixed number of suppliers from a VOR list – I don't know whether a framework agreement can work like that.

Possibly yes, but your description of "supplier list" seems as broad as that of a "multi-use list", i.e. the super-category itself? (E.g. a simple framework agreement with one tenderer would also be a "supplier list"?)

What I'm struggling with is where the lines are between these concepts :) I suspect that they are indeed in a conceptual hierarchy, with supplier list possibly at the top.

For a framework agreement, the line seems to be greater predictability as to how call-offs are awarded, as well as what the call-offs are about (the object of procurement), based on a pre-identified need of the buyer(s).

I don't know if there is a line between supplier lists and multi-use lists. I guess, at minimum, a multi-use list implies that the suppliers on the list are related in terms of what they provide, and that they had to be qualified to appear on the list, whereas – in the weakest sense – a supplier list could literally just be a list of suppliers (like, "local vendors", whether they mix cement or design websites) whose qualifications have not been verified.

As for qualification system, I think it's close to a multi-use list, and – based on my very limited understanding of QS – I don't know if there is a useful distinction to make between it and multi-use lists.

JachymHercher commented 2 years ago

That said, in most cases, the supplier list is used only once, in which case it is perhaps better to simply model this scenario the same as a restricted procedure in the EU – and only add additional processes in the exceptional cases.

If the supplier list is used only once, then indeed I would just deal with it as a standard restricted procedure. The tricky part about all of this seems to be the multi-use.

What I'm struggling with is where the lines are between these concepts :)

Worth reminding ourselves - there doesn't tend to be a sharp line, similarly as there isn't between FA and normal contracts. So I think we're looking rather for lines that are "good enough" in delimiting sets that are important for users.

I suspect that they are indeed in a conceptual hierarchy, with supplier list possibly at the top.

Concerning the hierarchy, I think the top-level should rather be the GPA's "multi-use lists", because GPA is the most general legal framework, no? National legislation is an implementation of the GPA, so all the other instruments (at least for above-threshold, which e.g. the VOR doesn't seem to be) must be some species of multi-use lists.

In Ontario, a buyer can do a limited procedure, inviting only a fixed number of suppliers from a VOR list – I don't know whether a framework agreement can work like that.

When reading this approach, I am imagining business needs (e.g. for a large nation-wide central purchasing body) such as:

You have a list of qualified potential suppliers that can build roads. When you build small roads, you contact the whole supplier list, but when you build really big roads, then you contact only the big & experienced companies from your list.
You have a list of qualified potential suppliers that can write studies. However, not all companies have experience/experts in all fields, so for each study, you want to only contact the companies that have them in the domain of the specific study.

Whether a FA can do that - hard to say. The business need seems clear, I don't see any breach of equal treatment or other procurement principles (e.g. you are able to ex-ante set the estimated/maximum value of the FA), and the way to make it happen also seems pretty straightforward - essentially having a second set of selection criteria for a specific mini-competition. So in theory, it could be fine. In practice, I'm not aware of anyone doing it and the law/eProcurement systems are not designed for it, so I'd guess they really aren't doing it.

Instead, what buyers sometimes do is that they set up a FA with separate lots which only differ in their selection criteria. Lot 1 selection criteria can be "yep, you're a building company" and lot 2 selection critera can be "yep, you're a building company big enough to build really big roads". As a buyer, you then use lots depending on what you need. The main difference compared to the VOR approach is that you have to know the "types" of selection criteria at the beginning of the procedure and design the lots appropriately. This would be a headache for the second example above, because there can be a huge variety of study domains and it would be more practical to do it on a "as needed" basis. Still, it seems plausible that this is rare enough that most jurisdictions don't care.

(All that being said, it's also possible that qualification systems do what VOR does, because you first qualify into the system and then buyers can run entire procedure with the qualified companies, which could include a more narrow set of selection criteria. But I've never found anyone with a good overview/understanding of qualification systems, so I don't know.)

jpmckinney commented 2 years ago

Thanks! We're getting clearer on where this fits. I like the suggestion to put "multi-use list" at the top of the conceptual hierarchy, inspired by the GPA being the highest-level legal instrument (as far as international agreements go).

That said, I don't think national legislation is an "implementation of" the GPA, but rather that national legislation is made to conform to it, in order to accede to the agreement. UNCITRAL is another high-level document, but it's more like a legislative template than a legal instrument. (It doesn't mention anything comparable.)

It's too bad we don't know anyone with a good overview/understanding of qualification systems. The procedures I know of in Canada and Ontario seem somewhat more ad-hoc (they don't set up explicit lots with selection criteria).

open-contracting / standard