Bid offer acceptance datasets are missing acceptances

CEardley commented 1 year ago

Describe the issue

BOAs are missing from BOALF endpoints. E.g., BOA 1193 for T_PINFB-3 on 2023-05-11.

Steps to Reproduce

Go to https://data.elexon.co.uk/bmrs/api/v1/datasets/BOALF?from=2023-05-11&to=2023-05-12&bmUnit=T_PINFB-3
Observe no BOA for T_PINFB-3 with acceptanceNumber 1193

Expected behaviour

Should return BOAs with acceptanceNumber = 1193 This BOA is seen in physical data on the Elexon platform https://www.bmreports.com/bmrs/?q=balancing/physicaldata (settlement period 8, 2023-05-11, T_PINFB-3) and detailed system price https://www.bmreports.com/bmrs/?q=balancing/detailprices (settlement period 8, 2023-05-11)

Observed behaviour

The below is the response from calling the URL listed above. Note 1193 for T_PINFB-3 is not present.

{
  "data": [
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 2,
      "settlementPeriodTo": 2,
      "timeFrom": "2023-05-11T23:57:00Z",
      "timeTo": "2023-05-11T23:58:00Z",
      "levelFrom": 0,
      "levelTo": -9,
      "acceptanceNumber": 1199,
      "acceptanceTime": "2023-05-11T23:36:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 2,
      "settlementPeriodTo": 2,
      "timeFrom": "2023-05-11T23:56:00Z",
      "timeTo": "2023-05-11T23:57:00Z",
      "levelFrom": 25,
      "levelTo": 0,
      "acceptanceNumber": 1199,
      "acceptanceTime": "2023-05-11T23:36:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 2,
      "settlementPeriodTo": 2,
      "timeFrom": "2023-05-11T23:41:00Z",
      "timeTo": "2023-05-11T23:56:00Z",
      "levelFrom": 25,
      "levelTo": 25,
      "acceptanceNumber": 1199,
      "acceptanceTime": "2023-05-11T23:36:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 2,
      "settlementPeriodTo": 2,
      "timeFrom": "2023-05-11T23:40:00Z",
      "timeTo": "2023-05-11T23:41:00Z",
      "levelFrom": 0,
      "levelTo": 25,
      "acceptanceNumber": 1199,
      "acceptanceTime": "2023-05-11T23:36:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 2,
      "settlementPeriodTo": 2,
      "timeFrom": "2023-05-11T23:39:00Z",
      "timeTo": "2023-05-11T23:40:00Z",
      "levelFrom": -9,
      "levelTo": 0,
      "acceptanceNumber": 1199,
      "acceptanceTime": "2023-05-11T23:36:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 1,
      "settlementPeriodTo": 1,
      "timeFrom": "2023-05-11T23:15:00Z",
      "timeTo": "2023-05-11T23:16:00Z",
      "levelFrom": 25,
      "levelTo": 0,
      "acceptanceNumber": 1198,
      "acceptanceTime": "2023-05-11T22:56:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-12",
      "settlementPeriodFrom": 1,
      "settlementPeriodTo": 1,
      "timeFrom": "2023-05-11T23:00:00Z",
      "timeTo": "2023-05-11T23:15:00Z",
      "levelFrom": 25,
      "levelTo": 25,
      "acceptanceNumber": 1198,
      "acceptanceTime": "2023-05-11T22:56:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 48,
      "settlementPeriodTo": 1,
      "timeFrom": "2023-05-11T22:59:00Z",
      "timeTo": "2023-05-11T23:00:00Z",
      "levelFrom": 0,
      "levelTo": 25,
      "acceptanceNumber": 1198,
      "acceptanceTime": "2023-05-11T22:56:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 44,
      "settlementPeriodTo": 44,
      "timeFrom": "2023-05-11T20:46:00Z",
      "timeTo": "2023-05-11T20:47:00Z",
      "levelFrom": 12,
      "levelTo": 0,
      "acceptanceNumber": 1197,
      "acceptanceTime": "2023-05-11T20:27:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 44,
      "settlementPeriodTo": 44,
      "timeFrom": "2023-05-11T20:31:00Z",
      "timeTo": "2023-05-11T20:46:00Z",
      "levelFrom": 12,
      "levelTo": 12,
      "acceptanceNumber": 1197,
      "acceptanceTime": "2023-05-11T20:27:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 44,
      "settlementPeriodTo": 44,
      "timeFrom": "2023-05-11T20:30:00Z",
      "timeTo": "2023-05-11T20:31:00Z",
      "levelFrom": 0,
      "levelTo": 12,
      "acceptanceNumber": 1197,
      "acceptanceTime": "2023-05-11T20:27:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 39,
      "settlementPeriodTo": 39,
      "timeFrom": "2023-05-11T18:23:00Z",
      "timeTo": "2023-05-11T18:24:00Z",
      "levelFrom": 14,
      "levelTo": 0,
      "acceptanceNumber": 1196,
      "acceptanceTime": "2023-05-11T18:13:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 39,
      "settlementPeriodTo": 39,
      "timeFrom": "2023-05-11T18:15:00Z",
      "timeTo": "2023-05-11T18:23:00Z",
      "levelFrom": 14,
      "levelTo": 14,
      "acceptanceNumber": 1196,
      "acceptanceTime": "2023-05-11T18:13:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 39,
      "settlementPeriodTo": 39,
      "timeFrom": "2023-05-11T18:15:00Z",
      "timeTo": "2023-05-11T18:16:00Z",
      "levelFrom": 14,
      "levelTo": 3,
      "acceptanceNumber": 1195,
      "acceptanceTime": "2023-05-11T18:01:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 39,
      "settlementPeriodTo": 39,
      "timeFrom": "2023-05-11T18:05:00Z",
      "timeTo": "2023-05-11T18:15:00Z",
      "levelFrom": 14,
      "levelTo": 14,
      "acceptanceNumber": 1195,
      "acceptanceTime": "2023-05-11T18:01:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 39,
      "settlementPeriodTo": 39,
      "timeFrom": "2023-05-11T18:04:00Z",
      "timeTo": "2023-05-11T18:05:00Z",
      "levelFrom": 3,
      "levelTo": 14,
      "acceptanceNumber": 1195,
      "acceptanceTime": "2023-05-11T18:01:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 6,
      "settlementPeriodTo": 6,
      "timeFrom": "2023-05-11T01:43:00Z",
      "timeTo": "2023-05-11T01:44:00Z",
      "levelFrom": 0,
      "levelTo": 3,
      "acceptanceNumber": 1192,
      "acceptanceTime": "2023-05-11T01:23:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 6,
      "settlementPeriodTo": 6,
      "timeFrom": "2023-05-11T01:42:00Z",
      "timeTo": "2023-05-11T01:43:00Z",
      "levelFrom": -6,
      "levelTo": 0,
      "acceptanceNumber": 1192,
      "acceptanceTime": "2023-05-11T01:23:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 5,
      "settlementPeriodTo": 6,
      "timeFrom": "2023-05-11T01:27:00Z",
      "timeTo": "2023-05-11T01:42:00Z",
      "levelFrom": -6,
      "levelTo": -6,
      "acceptanceNumber": 1192,
      "acceptanceTime": "2023-05-11T01:23:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 5,
      "settlementPeriodTo": 5,
      "timeFrom": "2023-05-11T01:26:00Z",
      "timeTo": "2023-05-11T01:27:00Z",
      "levelFrom": -4,
      "levelTo": -6,
      "acceptanceNumber": 1192,
      "acceptanceTime": "2023-05-11T01:23:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 4,
      "settlementPeriodTo": 4,
      "timeFrom": "2023-05-11T00:30:00Z",
      "timeTo": "2023-05-11T00:31:00Z",
      "levelFrom": 13,
      "levelTo": 0,
      "acceptanceNumber": 1191,
      "acceptanceTime": "2023-05-11T00:11:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 3,
      "settlementPeriodTo": 4,
      "timeFrom": "2023-05-11T00:15:00Z",
      "timeTo": "2023-05-11T00:30:00Z",
      "levelFrom": 13,
      "levelTo": 13,
      "acceptanceNumber": 1191,
      "acceptanceTime": "2023-05-11T00:11:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 3,
      "settlementPeriodTo": 3,
      "timeFrom": "2023-05-11T00:14:00Z",
      "timeTo": "2023-05-11T00:15:00Z",
      "levelFrom": 0,
      "levelTo": 13,
      "acceptanceNumber": 1191,
      "acceptanceTime": "2023-05-11T00:11:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    },
    {
      "dataset": "BOALF",
      "settlementDate": "2023-05-11",
      "settlementPeriodFrom": 3,
      "settlementPeriodTo": 3,
      "timeFrom": "2023-05-11T00:02:00Z",
      "timeTo": "2023-05-11T00:03:00Z",
      "levelFrom": -11,
      "levelTo": 0,
      "acceptanceNumber": 1190,
      "acceptanceTime": "2023-05-10T23:51:00Z",
      "deemedBoFlag": false,
      "soFlag": false,
      "amendmentFlag": "ORI",
      "storFlag": false,
      "rrFlag": false,
      "nationalGridBmUnit": "PINFB-3",
      "bmUnit": "T_PINFB-3"
    }
  ]
}

Screenshots

This screenshot is taken from the Physical Data page on BMRS for this BMU.

Additional context

T_PINFB-3 is one of 4 BMUs registered as part of the larger Capenhurst battery. BOA 1193 was issued to both T_PINFB-3 and T_PINFB-1 (as seen in detailed system price), however in the IRIS BOALF datasets it appears only for T_PINFB-1.

Checks

Please answer the following questions for yourself before submitting an issue.

[Y] I checked the documentation and found no answer
[Y] I checked to make sure that this issue has not already been filed
[Y] I'm reporting the issue to the correct repository
[Y] I'm reporting a problem and not a feature request
[Y] I have added the correct labels
[Y] I have added screenshots or other relevant evidence (if applicable)

beccar97 commented 1 year ago

Thanks for raising this, we're looking into what's happened here and why this data is missing. Will keep this updated as we know more and have an idea of when this will be resolved. If there are any other acceptances that you know are missing from the datasets please do let us know.

zghanty commented 1 year ago

@CEardley : Thank you for raising the issue. The team have identified the cause and have also followed up on the business rules wrt Acceptance numbers; this was an edge case as we did not foresee the same acceptance number for two different BMUs within the same Settlement Period.

ZakCannon commented 7 months ago

This issue has been resolved internally, and has been for a while. We're now looking at backfilling any data we missed as a result of this bug (for example, the mentioned when the issue was first raised), and will close this issue once we have

ZakCannon commented 7 months ago

This data has now been backfilled, and the data mentioned in the issue is now present at that endpoint. I'm going to go ahead and close this issue.

elexon-data / insights-issues