Handling multi-stage contracting processes

[timgdavies]

This issue builds on the review comments in #371, and discussions with the technical team and implementers, to set out an approach for OCDS 1.1 for handling multi-stage contracting processes, and sets a direction towards 2.0.

Current situation

The concept of a contracting process is central to OCDS. By bringing together multiple releases of data and information related to a single contracting process, OCDS enables citizens, companies and governments to get a clearer view of contracting activities.

The standard defines a contracting process as:

All the planning, tendering information, awards, contracts and contract implementation information related to a single initiation process.

Each contracting process is identified by an Open Contracting ID (ocid).

The standard currently contains a single tender block, and then arrays of one or more award and contract (and nested within contract, implementation) blocks.

This pattern, of a single tender block, was developed to avoid publication mistakes where tenders from different contracting processes could end up mixed together under the same ocid.

Limitations

The current model has major limitations in the case of contracting processes where multiple tender notices or solicitations are issued. This includes:

1. Two-stage processes pre-qualification shorlisting, and then a full invitation to tender;
2. Competitive dialogues - which may have repeated rounds;
3. Processes where the initial tender does not lead to an award, and so a second call for proposals is made;
4. Framework contracts with mini-competitions;

Initial proposal and review

In proposals for OCDS 1.1, we sought to address this by creating a relatedProcess block that could link two or more contracting processes. This would allow, for example, one contracting process to contain the pre-qualification tender, and award onto a shorlist, and a second, related process, to contain the invitation to qualified bidders and the final award of the contract.

However, peer-review feedback noted that this breaks the idea of bringing together all information about a contracting process under a single ocid.

The reviewer rejected the introduction of relatedProcess to deal with cases of frameworks, pre-qualification, competitive dialogue and other multi-stage processes.

We agree with the reviewer that the handling of pre-qualification and other multi-stage processes requires attention.

We believe that relatedProcess still has a role in some forms of framework, particularly cases where one agency establishes the framework, and other agencies are then running their own independent contracting processes to buy from that framework.

Alternative models and their implications

In this section we scope out alternative approaches to allow us to represent multi-stage contracting processes. This is guided by:

• Maintaining compatability with existing OCDS tools and data;
• Avoiding multiple ways of expressing the same thing;
• Balancing the desire of publishers to represent the complexities of their own procurement processes, with the needs of users for comparable data from across publishers, addressing a core set of common use-cases;
  • OCDS is a standard for transparent publication of contracting information: the core standard is not intended to capture the full business logic of specific contracting processes (although it may be extended, or building blocks re-used, in this way);

And based on a number of assumptions:

• Applications will use the content of the tender block to advertise available opportunities;
• Applications will use the contents of the award block to analyse firms in receipt of contracts, and the value of those contracts;
• Whilst the release model, and a versioned release, provides the ability to see the past states of a tender block, many applications will not be release-aware, and so will only look at the latest state of the tender.
• Given the persistence of releases, it is counter-intuitive for the single tender block of a contracting process to, at one time represent the 'qualification stage' of a process, and then at a later time, to change to represent the 'tender stage' of the process.
  • There are also risks that publishers would not correctly 'null' out values, creating a confused record of the tender processes.

Option (1): Convert tender into an array

Tender could become an array: possibly renamed to 'initiation' or 'solicitation', but using the same structure as the current Tender object. In this case:

• Each Tender object in the array would have a type to indicate whether it is a 'preQualification', 'tender' or some other solicitation as part of a single contracting process;
• Either:
  • Award and Bid (where the bid extesion is used) would need to have a tenderID property in order to make it clear which tender they related to (e.g. to distinguish awarding a supplier a place on the shortlist, from awarding them the contract); or
  • A separate 'shortlist' block (based on awards), and 'qualification' block based on bids would need to be introduced for qualified suppliers to be listed
• The relatedProcess cross-reference would need to be able to point to specific tender.id

This approach would not be backwards compatible, and so would need to be included in version 2.0, rather than 1.1.

Option (2): Create a distinct property for qualification

An additional qualification property could be added parallel to tender re-using the Tender object to represent the pre-qualification call, with tender then used for the call for proposals.

This approach does not change the semantics or structure of existing properties, so could be included in 1.1.

It works for two-stage processes, but does not work for processes with more than two stages.

It means that applications will need to look in qualification as well as tender to find opportunities to communicate to users.

The separate qualification object approach is taken in the PPP Implementation Profile of OCDS, although in this case, there is a lesser requirement that applications be able to advertise the qualification opportunity - as PPPs are lower volume - and the use-cases for the PPP Implementation Profile do not cover advertising opportunities.

Option (3): Re-use tender block for each stage

We would not change the schema, but would instead provide guidance that the tender object should be updated to reflect the most recent solicitation, wheter that was a qualification related notice, a tender, or a mini-competition.

The history of past states would be available from the versionedRelease and by looking at past releases.

However, this approach would:

(a) Make it difficult to see all the stages of a contracting process at once; (b) Risk creating a confused 'record' - as if the first qualification entry in a tender contains, for example, a set of items which are later removed from the full tender, and the subsequent tender does not 'null' these to remove them - the record will show a tender that included these items;

For this reason - we cannot recommend this option.

Related process for frameworks with competition

Based on the discussion in 310 we believe that it remains appropriate in some cases to have competitive call-offs from a framework modelled using separate contracting processes and relatedProcess.

This is summarised in the table below:

Framework type	OCDS approach
Single supplier with direct call-offs	A single contracting process using award(s) to represent the framework agreement and contract(s) to represent the call-offs.
Multiple suppliers with direct call offs	A single contracting process using award(s) to represent the framework agreement and contract(s) to represent the call-offs.
Multiple suppliers with mini-competitions for call-offs	Multiple contracting processes: One process using awards to represent suppliers on the framework agreement; Multiple selective or limited processes to represent the mini-competitions linked to the framework agreement via relatedProcess (see #371 ).
Multiple suppliers with either direct call-offs or mini-competitions	Multiple contracting processes: One process using awards to represent suppliers on the framework agreement and contract(s) to represent the direct call-offs; Multiple selective or limited processes to represent the mini-competitions linked to the framework agreement via relatedProcess (see #371 ).
Dynamic Purchasing System	Multiple contracting processes: One process using awards to represent suppliers joining the DPS. tender/status should be active for the lifetime of the dynamic purchasing system with tender/tenderPeriod and tender/awardPeriod reflecting that suppliers can join the DPS at any time. Multiple selective or limited processes to represent competitions between suppliers on the DPS for individual contracts, linked to the DPS via relatedProcess (see #371 ).

In the event that option (1) above is adopted, it would be possible for the mini-competitions that form part of a framework to be represented as part of a single contracting process.

However, there are cases where this is not appropriate (for example, when the framework is set up by a national agency, and assigned a process identifier in their national systems; and the competitive call-offs are carried out by local agencies, given distincy process identifiers in the local systems): so a choice would need to be made about whether to allow competitive frameworks to be modelled in those two different ways, or to restrict modelling of competitive framework call offs to the relatedProcess model.

Suggested actions

• Removing the 'preQualification', 'preSelection' codes from the relatedProcess codelist;
• Working up a proposal for version 2.0 to have an array of Tender objects, rather than a single tender entry;
• • Addressing the need for relatedProcess to specifically reference a tender.id;
• Working with publishers on interim approaches to model pre-qualification and multi-stage processes - though accepting that these are not fully supported in OCDS 1.1;

[JachymHercher]

Thanks a lot for the exhaustive overview. I don't have an opinion on the modeling options above (yet?), but please find below a few thoughts.

Concerning:

OCDS is a standard for transparent publication of contracting information: the core standard is not intended to capture the full business logic of specific contracting processes (although it may be extended, or building blocks re-used, in this way);

This is crucial. What do do we really need for transparent publication, and what can we ignore?

What definitely needs to be covered is what you need for submitting bids (i.e. the first round) and the results. What is between is a trade-off between more information for transparency and added complexity. In the EU, talking about multiple stage procedures, we do not really cover any of the intermediary steps - only the first stage and the result. (This could be phrased as an Option 0 above - we simply don't model this, because it's too complex.)

I'm not sure how much should be covered in general, but for example in case of competitive dialogues, I don't think that modeling all the intermediate stages adds much.

Concerning:

"We believe that relatedProcess still has a role in some forms of framework, particularly cases where one agency establishes the framework, and other agencies are then running their own independent contracting processes to buy from that framework."

and

Based on the discussion in 310 we believe that it remains appropriate in some cases to have competitive call-offs from a framework modelled using separate contracting processes and relatedProcess.

1) Besides disagreeing for the general reasons mentioned in #371, I think there is in fact a threat of inconsistency if you applied the multiple process approach only to frameworks.

For example, a restricted procedure with five lots is modeled as a single process. However, an open procedure with a framework agreement and five competitive call offs is, I would argue, quite similar, but would be modeled as 6 processes. The similarities: in both instances we start with a stage of "selecting a limited number of candidates" (even though the criteria differ); followed by five instances of "companies compete on award criteria for a "real" contract"; and, in both cases, there can be the same or different companies in each lot/call-off. Essentially, to make a brave analogy, I would argue that competitive call-offs are like lots (for call-offs you compete for the delivery of one thing at five different times, for lots you compete for five different things at one time), and so should involve the same number of processes.

2) I wouldn't mix up the two reasons for multiple processes that you mention above ("other agencies" and "competitive call offs").

Concerning other agencies, in EU procurement legislation, there are two different approaches to contracts (and in particular framework agreements) awarded by central purchasing bodies (a type of purchase by another agency). One is that buyers use framework agreements etc. established by central purchasing bodies (CPBs). The other one is that a CPB buys something and then transfers it to other public buyers.

In general, I would say the first option could be met by having the "buyer" section specified in the "contract" section (perhaps if it is different than the one in the buyer section). In other words, the procuringEntity runs the procedure, the Buyer is then per contract.

For the second option, in the EU, this is generally not public procurement. They are simply contracts (?) between public bodies. Is this type of relationship in the scope of OCDS?

Concerning:

Applications will use the contents of the award block to analyse firms in receipt of contracts, and the value of those contracts;

If we are thinking long-term, this may change - e.g. contractor could be moved to the contract section, not award, as discussed in #249

Finally, to add a twist, we've discussed this whole issue with @isarosa and, in fact, the Portuguese BASE models two stage procedures, call-offs etc. as separate (parent-child) processes - possibly similarly to what you proposed originally. The main reason is that while planning and implementation stages are not applicable for both of these processes (and winner and value for the contract stage mean something a bit different), the majority of e-procurement stages are the same: eNotification, eAccess, eSubmission, eEvaluation, eAward, (partially Contract). Going more into the actual experience of BASE could help resolve this issue.

(...perhaps both approaches can work, and it's mainly a question of which terminology is more intuitive - while ensuring consistency...)

[jpmckinney]

Relevant to EU forms.

[jpmckinney]

Relevant comments in #371 start from: https://github.com/open-contracting/standard/issues/371#issuecomment-292117603

[jpmckinney]

Note: https://github.com/eForms/eForms/issues/251#issuecomment-473663395

In eForms, the procedure/lot that "uses" a framework [to award contracts] is the same one that "establishes" it (only at a different time, i.e. in a later notice)

[jpmckinney]

Related: http://openeu.api-docs.openprocurement.org/en/latest/standard/qualification.html

[jpmckinney]

Related: https://github.com/open-contracting/standard/issues/784#issuecomment-510545905

[jpmckinney]

Summary regarding framework agreements with mini-competitions

The OCDS 1.1 guidance models the establishment of the framework agreement as one procedure (one OCID) and each mini-competition as a new procedure (new OCID). However, mini-competitions are part of the same procedure as the establishment of the framework agreement, in EU law and elsewhere. We therefore need a solution that represents these under one OCID, to respect the definition of 'contracting process' #866.

This issue (below) and issue #906 discuss specific proposals.

Additional problems with the OCDS 1.1 guidance are that:

• A different model is used for direct call-offs (one OCID) and mini-competitions (multiple OCIDs). Having a different model simply because a framework agreement reopens competition in its second stage seems arbitrary.
• In the case of mini-competitions, the contracting process for establishing the framework agreement has no possibility of containing contracts. This seems bizarre, especially if the framework agreement promises a minimum volume of orders.
• A mini-competition uses the tender block to describe the items to be procured. This block has the semantics of the 'first stage' of a procedure, but mini-competitions occur in a second stage. As such, the semantics are incorrect.
• A mini-competition (in a closed framework agreement) is closed to new suppliers. To indicate a closed procedure, procurementMethod is typically set to 'limited' or 'direct'. However, the correct code for a mini-competition is 'selective' (per GPA definitions), since the participants were selected in the first stage of the procedure (i.e. in the setup of the framework agreement). As such, potential suppliers will have a more difficult time identifying opportunities.

Discussion

Responding to the above, now that I'm looking at the EU forms yet again, I think:

1. Two-stage processes pre-qualification shorlisting, and then a full invitation to tender

I don't see a problem with option 2 (new block adjacent to tender). Regarding:

applications will need to look in qualification as well as tender to find opportunities to communicate to users

To clarify, the issue with the proposed model is that qualification is the block that describes the request to participate, which is the first stage and therefore of interest for finding opportunities.

The issue is easily resolved if the definition of tender is adjusted to mean "first stage" in all cases. Right now the tender field is described as:

The activities undertaken in order to enter into a contract.

This gives sufficient latitude to adjust semantics. The Tender definition is described as:

Data regarding tender process - publicly inviting prospective contractors to submit bids for evaluation and selecting a winner or winners.

This gives less latitude. However, these semantics are already widely ignored if interpreted strictly, such as for framework agreements.

If tender means "first stage," it remains the single place for finding opportunities (in this case, the request to participate), eliminating one of two complaints with this approach (the other is addressed below). We can then use a secondStage block for the request for tender.

2. Competitive dialogues - which may have repeated rounds

I have yet to see data about these rounds. For now, we can pursue @JachymHercher's option 0. In future, I don't see an issue with secondStage as above being an array to contain each round.

3. Processes where the initial tender does not lead to an award, and so a second call for proposals is made

Some contexts consider this scenario to be one contracting process, others consider it to be two. For OCDS to be consistent, we ought to choose one model. My preference is two.

This is also consistent with the concept that tender means "first stage". If you have a second call for proposals as in the example, you are having a "first stage" for a second time.

4. Framework contracts with mini-competitions

It seems consistent with the above to use the proposed secondStage array. Having a different model simply because a framework agreement reopens competition seems arbitrary. Having a contracting process (framework agreement set-up) with no possibility of containing contracts seems bizarre. Having a "first stage" (in a mini-competition) that is in fact a second stage and that is closed to competition (even though it is technically a 'selective' procedure) seems to deeply break the desirable semantics of OCDS.

Proposal

My recommendation would be to work on a proposal for a secondStage extension, which would likely be merged into OCDS, considering only open procedures and direct awards have one stage.

We would need to:

• deprecate the framework code in relatedProcess.csv
• update the framework agreement guidance, including the in-progress PR #890 cc @mrshll1001 @duncandewhurst

• deprecate the qualification extension since it has the issues with the original model for option 2

This proposal, in short, is like option 1 (changing tender into an array). However, option 1 loses the distinction between first and second stage. Preserving the distinction is important, as it prevents implementers from accidentally putting multiple first stages into an array.

@JachymHercher's 'brave analogy' between lots and call-offs seems appropriate: lots are the way to multiply components of a first stage. Having a secondStage array would allow OCDS to multiply components of a second stage (dialogue rounds, mini-competitions, etc.).

[yolile]

Second Stage extension drafted here https://gitlab.com/dncp-opendata/ocds_secondStage_extension

And some comments from discussions/retreats:

• We need a field to distinguish the FA and prequalification tender from the traditional ones: use the techniques extension https://github.com/open-contracting-extensions/ocds_techniques_extension
• We need a field to distinguish the mini competition tenders from the direct call offs: no invitations array in secondStage object
• Plannings in FAs mini competitions: use the budgetBreakdown extension and add the budget lines here for each purchase
• Distinguish direct and competitive awards: the direct ones dont have the 'invitationID' field.
• The extension also adds a 'documents' array in the secondStage object to disclose the documents related to the second stage, as the award document for the tender stage, etc
• For indicate what award if for what buyer, use the contracts.buyer extension, that will now be award.buyer extension
• For mapping the award and contract block for direct call offs follow the same rules that are for the 1.1.5 version

cc @jpmckinney

[jpmckinney]

@PaulBoroday The extension above is the proposal for OCDS 1.2 (i.e. backwards-compatible with OCDS 1.1). In short, it adds a secondStage object, which has fields for the candidates who were qualified to advance to the second stage, and for the invitations (e.g. requests for tender) that occur at the second stage. The Invitation object is very similar to the Tender object with only a few differences. The extension is intended to support any two-stage procedure, including framework agreements. (You can ignore some of the Paraguay-specific fields in the extension like simultaneousSupply; these fields are not relevant to the modelling for framework agreements.)

[jpmckinney]

Adding resources to review/consider:

• The Law and Economics of Framework Agreements: Designing Flexible Solutions for Public Procurement
• Handbook of Procurement
• EU Public Contract Law: Public Procurement and Beyond (published prior to 2014 laws, though)

[PaulBoroday]

Hi @jpmckinney ! How it is now considering that we have secondStage in the EU extension with a bit different logic?

[jpmckinney]

The EU extension is just about describing the second-stage from the perspective of the first stage (e.g. min/max candidates): https://extensions.open-contracting.org/en/extensions/secondStageDescription/master/ The EU legislation / process doesn't describe the second-stage itself (the specific invitations to bid); it only announces the awards from any mini-competitions.

The DNCP extension above is instead about modelling the second-stage itself (the invitations, the bidders, etc.).

[jpmckinney]

UNCITRAL glossary defines:

ID	Term	Definition	Reference
74	Second-stage competition	A stage in closed framework agreements with more than one supplier or contractor and in open framework agreements through which certain terms and conditions of the procurement that cannot be established with sufficient precision when the framework agreement is concluded are established or refined through a competition between or among suppliers or contractors parties to the framework agreement. For the explanation of the terms “closed framework agreement”, “open framework agreement”, “procurement” and “framework agreement”, see ## 10, 48, 58 and 31 above. For the explanation of the term “supplier or contractor”, see # 85 below.	N/A
88	Two-stage tendering	A method of procurement and one of the forms of tendering, which main distinct feature is two-stage process: - the first stage involves the discussion between the procuring entity and suppliers or contractors of various aspects of their initial tenders excluding price, in order to refine aspects of the description of the subject matter of the procurement and to formulate them with the detail required under article 10 of the Model Law; and - the second stage involves submission of final tenders with price in response to the revised set of terms and conditions of the procurement, examination and evaluation of final tenders and award of the procurement contract. For the explanation of the terms “method of procurement”, “procuring entity”, “supplier or contractor”, “initial tenders”, “description of the subject matter of the procurement”, “procurement”, “examination”, “evaluation” and “award of a procurement contract”, see ## 44, 62, 85, 40, 18, 58, 29, 27 and 5 above.	two-stage bidding procedure (the World Bank procurement guidelines)

EBRD glossary lists:

UNCITRAL Model Law Terms	Description Notes	Others in use or similar Terms
Second stage competition	Competition to award contracts under a framework agreement	Call off (EU)

[duncandewhurst]

The World Bank recently published a Guidebook for Setting-up and Operating Framework Agreements. From a quick scan, it mostly references familiar sources: the EU Directives, the UNCITRAL model law and The Law and Economics of Framework Agreements book. However, it contains seven country case-studies which may be of interest.