awardCriteria, techniques: Modelling relationships to lots

[jpmckinney]

For https://github.com/open-contracting-extensions/ocds_awardCriteriaBreakdown_extension, we allow a set of award criteria to be associated to one or more lots. This makes sense, as I anticipate that multiple lots in the same procedure might have the same award criteria.

For https://github.com/open-contracting-extensions/ocds_techniques_extension, the fields were originally on Tender, but I moved them to Lot, as they can differ per lot. I assumed that the details of FAs, DPS's and electronic auctions are unlikely to be shared across lots. (For example, I wouldn't expect an auction URL to be shared.) And even if shared, the amount of repeated data is minimal. Similarly, for min/max candidates and rationale, I assume this is more relevant on the individual lot.

If we were to strictly follow the eForms model, then we'd have many arrays of objects in Tender, each with a relatedLots field. This would minimize repetition, but it would also make it much more difficult to gather information about one lot (you'll have to loop through all the arrays to check if the lot id is in relatedLots).

If we instead embraced repetition, we might end up with very large JSON files. So:

1. Should we (a) avoid repetition, (b) embrace repetition or (c) find a balance?
2. If (c), what is the right balance (at least for the cases above)?

[jpmckinney]

In TED, award criteria are repeated per lot, so perhaps we should just repeat? It would avoid the somewhat awkward awardCriteriaBreakdown container.

[ColinMaudry]

I guess we're hitting the inherent limit of a tree-shaped data format vs a graph that natively allows many-to-many relationships between the objects. I'm consequently not sure how we could find a clean balance (c)).

Overall, if it's a) or b)I believe the best approach is repetition b), as it the most user-friendly approach. However, there is a use case that I wish to be covered, it is the possibility to check which lots have the same bidding conditions, without checking condition by condition.

Ideally, the data would offer both lot and conditions oriented information. Isn't it something the merge tool could do? Aren't the records already a "built" version of "source" releases?

[ColinMaudry]

Could we give local ids to sets of conditions, not for referencing, but to indicate that if condition sets ids are identical, the conditions are identical?

Problem: wouldn't it be tricky for data producers to set this id. Would their data know where ids should be the same?

[jpmckinney]

Isn't it something the merge tool could do? Aren't the records already a "built" version of "source" releases?

It terms of easing use, we can think of many transformations that tailor the OCDS format to more easily answer specific questions. For this issue, we should decide what is the 'base' format (which will be a compromise, as we're not using linked data).

there is a use case that I wish to be covered, it is the possibility to check which lots have the same bidding conditions, without checking condition by condition.

This is a good use case. "if an object would be repeated, (a) would I want to be able to find all repetitions and (b) how difficult is it to find them?" I think this means it is better to not repeat award criteria, but okay to repeat techniques and candidate limits.

Could we give local ids to sets of conditions, not for referencing, but to indicate that if condition sets ids are identical, the conditions are identical?

I think this would be tricky, yes. It might be easier for code to determine when two objects are the same (we're developing this code for quality checks, so we can share it when ready.)

[jpmckinney]

On the other hand, if you want to find similar/same conditions across contracting processes, then you still need to compare objects. This seems to me to be at least as common a use case as finding similar conditions within the same process. Perhaps, then, it is simplest to always repeat?

[ColinMaudry]

Repeating is a "dumber" approach, which keeps complexity low and makes the data more accessible. Let's repeat the conditions across lots.

That means we would have sets of properties/objects (techniques, awardCriteria, selection criteria, submission techniques (electronic auction, etc.) that could be used either at Tender or at Lot level.

[jpmckinney]

Repeating is a "dumber" approach, which keeps complexity low and makes the data more accessible. Let's repeat the conditions across lots.

Sounds good.

That means we would have sets of properties/objects (techniques, awardCriteria, selection criteria, submission techniques (electronic auction, etc.) that could be used either at Tender or at Lot level.

My proposal is to avoid putting any more of these at Tender level, and to instead move towards always having at least one Lot.

[ColinMaudry]

At tender level it's a single lot procedure At lot level if it's a multi-lot procedure.

Otherwise, we could always store these conditions at lot level, even in single lot scenarios (as eForms seems to do), but I fear it would be a radical change compared with existing OCDS practices. Would it?

[ColinMaudry]

(the comment above crossed yours :))

[jpmckinney]

It would be a significant change, yes; there is an issue proposing the change for 1.2.0: https://github.com/open-contracting/standard/issues/891 That issue describes the reasons why the change is important to make.

Since these extensions will be primarily used in the EU (or in other jurisdictions with lots), I think we can just extend Lot for now, as using lots is not inconsistent with OCDS.

[ColinMaudry]

Since the awardCriteriaBreakdown extension (to be renamed awardCriteria extension) adopted a per-lot structure, can we close this issue?

[jpmckinney]

Yes, thanks!