FREQ report is littered with erroneous zero-values for ~25 hours in mid June

[Samreay]

Describe the issue

Rolling frequency data is polluted with erroneous zero values from around 5pm 2023-06-10 to 9pm 2023-06-11

Steps to Reproduce

Steps to reproduce the behavior:

1. Go to https://bmrs.elexon.co.uk/api-documentation/endpoint/datasets/FREQ/stream
2. Set from time to 2023-06-11T00:00Z
3. Set to time to 2023-06-11T01:00Z
4. Behold the chaos

Expected behaviour

Frequency should never drop to zero. Zero values should be caught and raised internally as data issues and be withheld from publication.

Observed behaviour

Tons of zeros sprinkled everywhere.

{
    "dataset": "FREQ",
    "measurementTime": "2023-06-11T00:54:00Z",
    "frequency": 49.865
  },
  {
    "dataset": "FREQ",
    "measurementTime": "2023-06-11T00:53:45Z",
    "frequency": 0
  },
  {
    "dataset": "FREQ",
    "measurementTime": "2023-06-11T00:53:30Z",
    "frequency": 49.882
  },

Screenshots

Checks

Please answer the following questions for yourself before submitting an issue.

• [x] I checked the documentation and found no answer
• [x] I checked to make sure that this issue has not already been filed
• [x] I'm reporting the issue to the correct repository
• [x] I'm reporting a problem and not a feature request
• [x] I have added the correct labels
• [x] I have added screenshots or other relevant evidence (if applicable)

[zghanty]

Hello Sam, this look an upstream issue, I will raise this with NGESO

[zghanty]

Hi Sam, NGESO are still looking into this. Sorry for the delay

[Samreay]

No worries mate. Let me know if there's a way I can help prod this on NGESOs side as well :)

[caligoig]

Hello, I see this issue still hasn't been resolved. I have added a list of all dates for entire FREQ history where frequency is set to 0. File FREQ_0s.csv What is worrying is there are zeros in most recent entries as well March 2024 (scroll to the bottom of the csv file attached) Can you please let us know when you expect issue will be resolved. Thanks FREQ_0s.csv

[Kim-Balmer]

Could you please advise with API you used. Looking at https://bmrs.elexon.co.uk/api-documentation/endpoint/system/frequency https://bmrs.elexon.co.uk/api-documentation/endpoint/datasets/FREQ/stream I am unable to replicate your example and values are shown.

[caligoig]

Hey @Kim-Balmer These are few links samples you can replicate my issues with

https://data.elexon.co.uk/bmrs/api/v1/datasets/FREQ/stream?measurementDateTimeFrom=2023-06-10T17%3A57%3A45Z&measurementDateTimeTo=2023-06-10T17%3A57%3A45Z https://data.elexon.co.uk/bmrs/api/v1/datasets/FREQ/stream?measurementDateTimeFrom=2023-12-14T10%3A11%3A45Z&measurementDateTimeTo=2023-12-14T10%3A11%3A45Z https://data.elexon.co.uk/bmrs/api/v1/datasets/FREQ/stream?measurementDateTimeFrom=2024-03-12T13%3A18%3A00Z&measurementDateTimeTo=2024-03-12T13%3A18%3A30Z Links above only display 0 values

https://data.elexon.co.uk/bmrs/api/v1/datasets/FREQ/stream?measurementDateTimeFrom=2023-12-10T00%3A01%3A45Z&measurementDateTimeTo=2023-12-10T00%3A11%3A45Z Here you can see 0 followed by few normal values, then it displays 0 again, followed by few normal values again, and 0 again and so on

Let me know if you need anything else Thanks for looking into this

[Samreay]

Yeah I can confirm we've still been seeing this too: