open-contracting / european-union-support

Support scripts for TED mapping
BSD 3-Clause "New" or "Revised" License
9 stars 3 forks source link

eForms expects users to look up the value of the winning bid #178

Closed jpmckinney closed 1 year ago

jpmckinney commented 1 year ago

There's no field on the SettledContract, LotResult, etc. for the award or contract value.

As far as I understand, a user is expected to lookup the winning bid, whose value is set by BT-720-Tender.

Can/should we update the OCDS mapping to set the award or contract value, to be more easily accessible?

odscjen commented 1 year ago

Yes I think we should. It can be additional instructions at BT-720-Tender.

The relevant xml in the eForms is (taken from can_24_maximal.xml):

<efac:LotResult>
    <cbc:ID schemeName="result">RES-0002</cbc:ID>
    <efac:LotTender>
        <cbc:ID schemeName="tender">TEN-0001</cbc:ID>
    </efac:LotTender>
    <efac:TenderLot>
        <cbc:ID schemeName="Lot">LOT-0002</cbc:ID>
    </efac:TenderLot>
</efac:LotResult>
<efac:LotTender>
    <cbc:ID schemeName="tender">TEN-0001</cbc:ID>
    <cac:LegalMonetaryTotal>
        <cbc:PayableAmount currencyID="EUR">9999999.99</cbc:PayableAmount>
    </cac:LegalMonetaryTotal>
    <efac:TenderLot>
        <cbc:ID schemeName="Lot">LOT-0002</cbc:ID>
    </efac:TenderLot>
</efac:LotTender>

Where efac:LotResult/cbc:ID is the awards.id, efac:LotTender/cbc:ID is the bids.id and efac:TenderLot:cbc:ID is the lots.id. cac:LegalMonetaryTotal is BT-720-Tender.

I don't think any of the current common guidance in operations.md will fit. An award notice can have multiple LotResults each of which references a LotTender and a TenderLot.

Current guidance:

Get the bid for the LotTender and map to the bid''s .value. Map @currencyID to the value''s .currency.

Additional guidance:

Get the ancestor::efac:NoticeResult/efac:LotResult with an /efac:LotTender/cbc:ID equal to the value of ancestor::efac:LotTender/cbc:ID. Get the Award in awards whose id is equal to the value of this efac:LotResult's /cbc:ID. If none exists yet:

  1. Add an Award to awards
  2. Set its .id to the value of this efac:LotResult/cbc:ID

Add the value of ancestor::efac:TenderLot/cbc:ID to the award's .relatedLots Add the value of this field to the awards .value.amount and map @currencyID to the value's `.currency

A chunk of this is very similar to Get the award for a LotResult but I don't think we can just use that common guidance as it references ancestor::efac:LotResult/cbc:ID which is ambiguous from BT-720 as there can be numerous fields that fit that XPATH.

odscjen commented 1 year ago

On a related note I wonder if it is worth taking the time to try and map out an entire sample eForms notice into OCDS following the guidance we've got so far. I suspect we might spot a few missing pieces of OCDS data or confusions. But it'll be quick a hefty task to do so may not be worth the time?

jpmckinney commented 1 year ago

I wonder if it is worth taking the time to try and map out an entire sample eForms notice

We didn't do that for the EU profile, but we did identify some things to change or fix once the UK and Scotland started implementation. I think most of the changelog is based on implementation lessons.

It's maybe feasible to try and map the "minimal" examples:

This should cover all the "core" concepts like bids, awards, contracts, etc. – so we would identify issues like awards having no values. The more complete examples are less relevant, as the worst case for an extension field is that the mapping doesn't work (I don't suspect any would map to the wrong thing).

To many +map+ one of those minimal notices, how long do you think it would take?

If we were to map more complete examples, it'd be with:

duncandewhurst commented 1 year ago

All sounds good to me, with one change to the final sentence int he additional guidance:

Add Map the value of this field to the awards .value.amount and map @currencyID to the value's `.currency

odscjen commented 1 year ago

To many one of those minimal notices, how long do you think it would take?

It's hard to say, I suspect that as long as there were no issues mapping one of the minimal notices could take up to a day. Obviously if any issues with the guidance were discovered along the way this would add to the time, possibly by as much as an extra day or two?

odscjen commented 1 year ago

I'll update the guidance for BT-720-Tender incorporating Duncan's update.

jpmckinney commented 1 year ago

Okay, let's try mapping https://github.com/OP-TED/eForms-SDK/blob/develop/examples/notices/can_24_minimal.xml and see what we learn from the exercise.

jpmckinney commented 1 year ago

cc @camilamila, we're updating the eForms profile to fill in award/contract value, instead of requiring users to look up the winning bid like in eForms (the winning bid will still be there in the OCDS, of course).

odscjen commented 1 year ago

I mapped out can_24_minimal based on the current draft guidance.yaml and operations.md. I then ran the resulting json through the Data Review Tool (for which I added some dummy release package fields) and it was valid OCDS at least! There were a few mostly minor problems though so I'll open up separate issues for those.

{
  "id": "65667997-031a-4198-bd25-7225449ef479",
  "initiationType": "tender",
  "ocid": "blah",
  "date": "2019-10-23T00:00:00+01:00",
  "tag": [
    "award",
    "contract"
  ],
  "language": "FR",
  "parties": [
    {
      "id": "ORG-0004",
      "name": "Espaces Verts Lemire",
      "roles": [
        "tenderer",
        "supplier"
      ],
      "address": {
        "locality": "Le Grand-Quevilly",
        "postalCode": "ABC123",
        "region": "FRD22",
        "country": "FR"
      },
      "details": {
        "listedOnRegulatedMarket": false,
        "scale": "micro"
      },
      "beneficialOwners": [
        {
          "id": "UBO-0001",
          "nationality": "FR"
        }
      ],
      "identifier": {
        "id": "326 912 557"
      },
      "contactPoint": {
        "telephone": "+33 2 35 67 82 82",
        "email": "contact@example.com"
      }
    },
    {
      "id": "ORG-0001",
      "name": "Rouen Habitat",
      "roles": [
        "buyer"
      ],
      "details": {
        "classifications": [
          {
            "id": "pub-undert-la",
            "description": "Public undertaking, controlled by a local authority",
            "scheme": "TED_CA_TYPE"
          },
          {
            "description": "hc-am",
            "scheme": "COFOG",
            "id": "06"
          }
        ]
      },
      "address": {
        "locality": "Rouen Cedex 1",
        "postalCode": "ABC123",
        "region": "FRD22",
        "country": "FR"
      },
      "identifier": {
        "id": "123 456 789"
      },
      "contactPoint": {
        "telephone": "+33 235156161",
        "email": "contact@rouenhabitat.fr"
      }
    },
    {
      "id": "ORG-0002",
      "name": "Tribunal administratif de Rouen",
      "address": {
        "locality": "Rouen",
        "postalCode": "ABC123",
        "region": "FRD22",
        "country": "FR"
      },
      "identifier": {
        "id": "123 456 789"
      },
      "contactPoint": {
        "telephone": "+33 232081270",
        "email": "greffe.ta-rouen@juradm.fr"
      },
      "roles": [
        "reviewBody"
      ]
    },
    {
      "id": "ORG-EU",
      "name": "European Union",
      "roles": [
        "funder"
      ]
    }
  ],
  "bids": {
    "details": [
      {
        "id": "TEN-0001",
        "rank": 1,
        "hasRank": true,
        "value": {
          "amount": 95414,
          "currency": "EUR"
        },
        "hasSubcontracting": false,
        "tenderers": [
          {
            "id": "ORG-0004"
          }
        ],
        "relatedLots": [
          "LOT-0000"
        ]
      }
    ],
    "statistics": [
      {
        "id": "1",
        "measure": "highestValidBidValue",
        "value": {
          "amount": 95414,
          "currency": "EUR"
        },
        "relatedLots": [
          "LOT-0000"
        ]
      },
      {
        "id": "2",
        "measure": "lowestValidBidValue",
        "value": {
          "amount": 59299.64,
          "currency": "EUR"
        },
        "relatedLots": [
          "LOT-0000"
        ]
      },
      {
        "id": "3",
        "measure": "bids",
        "value": 3,
        "relatedLots": [
          "LOT-0000"
        ]
      }
    ]
  },
  "tender": {
    "id": "022d0ef9-6338-42d9-afa4-a87709128061",
    "title": "Service d'entretien de remise en état et de nettoyage des espaces verts",
    "description": "Service d'entretien de remise en état et de nettoyage des espaces verts.",
    "mainProcurementCategory": "services",
    "items": [
      {
        "id": 1,
        "classifications":[
          {
            "id": "77310000",
            "scheme": "CPV"
          }
        ]        
      }
    ],
    "legalBasis": {
      "id": "32014L0024",
      "scheme": "CELEX"
    },
    "status": "complete",
    "procurementMethod": "open",
    "lots": [
      {
        "id": "LOT-0000",
        "title": "Agence centre",
        "description": "Service d'entretien de remise en état et de nettoyage des espaces verts.",
        "awardCriteria": {
          "weightingDescription": "Le calcul du score prix-qualité est basé sur …",
          "criteria": [
            {
              "type": "price",
              "description": "Le prix contribue for 40 % …"
            }
          ]
        },
        "mainProcurementCategory": "services",
        "items": [
          {
            "id": "2",
            "classifications": [
              {
                "scheme": "CPV",
                "id": "77310000"
              }
            ]
          }
        ],
        "reviewDetails": "Un appel peut...",
        "coveredBy": [
          "GPA"
        ],
        "techniques": {
          "hasElectronicAuction": true
        },
        "identifiers": {
          "id": "1",
          "scheme": "internal"
        }
      }
    ]
  },
  "awards": [
    {
      "id": "RES-0001",
      "status": "active",
      "statusDetails": "At least one winner was chosen.",
      "buyers": [
        {
          "id": "ORG-0001"
        }
      ],
      "suppliers": [
        {
          "id": "ORG-0004"
        }
      ],
      "value": {
        "amount": 95414,
        "currency": "EUR"
      },
      "relatedLots": [
        "LOT-0000"
      ],
      "relatedBids": [
        "TEN-0001"
      ]
    }
  ],
  "contracts": [
    {
      "id": "CON-0001",
      "awardID": "RES-0001",
      "dateSigned": "2019-09-03T00:00:00+02:00",
      "identifiers": [
        {
          "id": "SCN DEF:2020-12",
          "scheme": "FR-SCN DEF"
        }
      ],
      "relatedBids": [
        "TEN-0001"
      ]
    }
  ]
}
odscjen commented 1 year ago

@jpmckinney I'm happy with how the mapping of can_24_minimal went, I don't think there are any key fields not mapped to, can we close this issue or do you think there's more to do here?

jpmckinney commented 1 year ago

Yes, the release above looks good. Thank you!