Period: None in B1610 return value

[BenPortner]

Hi @MichaelKavanagh,

it's me again. 😅 Just tried elexon v0.1.1 B1610 API with v2 parameter as you suggested in #2. There seems to be a problem with the internal result parser because the period field remains empty.

My code:

from elexon import ElexonRawClient
api = ElexonRawClient(apikey, 'v2')
generation = api.Transparency.B1610(SettlementDate = '2020-01-01', Period = '*')

generation is an array with entries like this:

{'documentType': 'Actual generation',
 'businessType': 'Production',
 'processType': 'Realised',
 'timeSeriesID': 'ELX-EMFIP-AGOG-TS-18481',
 'curveType': 'Sequential fixed size block',
 'settlementDate': datetime.datetime(2020, 1, 1, 0, 0),
 'powerSystemResourceType': 'Generation',
 'registeredResourceEICCode': '48W000000DAMC-1M',
 'marketGenerationUnitEICCode': '48W000000DAMC-1M',
 'marketGenerationBMUId': 'T_DAMC-1',
 'marketGenerationNGCBMUId': 'DAMC-1',
 'bMUnitID': 'T_DAMC-1',
 'nGCBMUnitID': 'DAMC-1',
 'activeFlag': True,
 'documentID': 'ELX-EMFIP-AGOG-22495386',
 'documentRevNum': 1,
 'Period': None}

Unfortunately, the Period field is empty for all array elements. When I do the API call via browser, I see that Period looks like this:

<Period>
<timeInterval>
<start>2020-01-01</start>
<end>2020-01-01</end>
</timeInterval>
<resolution>PT30M</resolution>
<Point>
<settlementPeriod>48</settlementPeriod>
<quantity>280.8</quantity>
</Point>
<Point>
<settlementPeriod>47</settlementPeriod>
<quantity>154.44</quantity>
</Point>
<Point>
<settlementPeriod>46</settlementPeriod>
<quantity>71.28</quantity>
</Point>
</Period>

Which makes me assume that the Period xml is not parsed properly by elexon?

Keep up the great work 😉

Ben

[michaelkavanagh]

sigh thanks for flagging this up.

Looks like this needs some recursion. (Un-)helpfully the documentation still gives the old structure...

This will take some thought.

[BenPortner]

Suggestion for a quick and dirty workaround: use ServiceType=csv and parse the result into a list?

[michaelkavanagh]

Yes, csv is returned by the wrapper as a raw string at the moment (see elexon.py:181) and appears to work for this endpoint. Use csv service type by instantiating:

api = ElexonRawClient(api_key = 'API_KEY', api_version = 'v2', api_service_type = 'csv')

[michaelkavanagh]

@BenPortner I think I have come up with a solution to this but it is a big change. Are you able to clone the source from GitHub and help test it? If so I can push the branch to GitHub.

I am reluctant to just push it without making sure it doesn't break the old (mostly correct) behaviour on the other report endpoints.

[BenPortner]

Hi @MichaelKavanagh,

I don't really know anything about the other endpoints, but I could test B1610 :) would that help?

Ben

[michaelkavanagh]

@BenPortner yes, if you could pull the B1610 branch and test that would be great. Thanks.

[BenPortner]

Just opened a PR for the first test. The new branch looks good so far. I noticed only two little inconsistencies:

Sample code:

from elexon import ElexonRawClient
api = ElexonRawClient(my_key, 'v2')
generation = api.Transparency.B1610(SettlementDate='2020-01-01', Period='2', NGCBMUnitID='CAS-BEU01')

Sample output:

{'documentType': 'Actual generation',
 'businessType': 'Production',
 'processType': 'Realised',
 'timeSeriesID': 'ELX-EMFIP-AGOG-TS-583',
 'curveType': 'Sequential fixed size block',
 'settlementDate': datetime.datetime(2020, 1, 1, 0, 0),
 'powerSystemResourceType': 'Generation',
 'registeredResourceEICCode': '48W000CAS-BEU01F',
 'marketGenerationUnitEICCode': '48W000CAS-BEU01F',
 'marketGenerationBMUId': 'M_CAS-BEU01',
 'marketGenerationNGCBMUId': 'CAS-BEU01',
 'bMUnitID': 'M_CAS-BEU01',
 'nGCBMUnitID': 'CAS-BEU01',
 'activeFlag': True,
 'documentID': 'ELX-EMFIP-AGOG-22495386',
 'documentRevNum': 1,
 'Period': {'timeInterval': {'start': datetime.datetime(2020, 1, 1, 0, 0),
   'end': datetime.datetime(2020, 1, 1, 0, 0)},
  'resolution': 'PT30M',
  'Point': {'settlementPeriod': 2, 'quantity': 17.172}}}

1) Notice fields nGCBMUnitID and bMUnitID have lower-case first letters for some reason. Period has a capital letter on the other hand. Should capitalization be more consistent? Your choice 😉 2) Period.timeInterval.start/end present the correct date but not the correct time (settlement period 2 should be 00:30 till 01:00). This is a minor issue but I believe if the fields would implement the time correctly than that would be a huge usability benefit for users.

[michaelkavanagh]

1. They are as-returned by the Elexon BMRS api. Since this is the 'raw' client, I think these should be kept true to the response.

2. The response from Elexon is just the date, no time information:

   <timeInterval>
   <start>2020-01-01</start>
   <end>2020-01-01</end>
   </timeInterval>

   Therefore I have removed the time component (which was incorrectly added by the parser) in 7a8b485ebe6292d5e36e854964d275fa74ec6a3e. The returned data is now:

   {
   'documentType': 'Actual generation',
   'businessType': 'Production',
   'processType': 'Realised',
   'timeSeriesID': 'ELX-EMFIP-AGOG-TS-583',
   'curveType': 'Sequential fixed size block',
   'settlementDate': datetime.date(2020, 1, 1),
   'powerSystemResourceType': 'Generation',
   'registeredResourceEICCode': '48W000CAS-BEU01F',
   'marketGenerationUnitEICCode': '48W000CAS-BEU01F',
   'marketGenerationBMUId': 'M_CAS-BEU01',
   'marketGenerationNGCBMUId': 'CAS-BEU01',
   'bMUnitID': 'M_CAS-BEU01',
   'nGCBMUnitID': 'CAS-BEU01',
   'activeFlag': True,
   'documentID': 'ELX-EMFIP-AGOG-22495386',
   'documentRevNum': 1,
   'Period': {
       'timeInterval': {
           'start': datetime.date(2020, 1, 1),
           'end': datetime.date(2020, 1, 1)
       },
       'resolution': 'PT30M',
       'Point': {
           'settlementPeriod': 2,
           'quantity': 17.172
       }
   }
   }

I believe any further processing should be done in a subclass of ElexonRawClient.

I have also done a bit of testing over the weekend, so if you don't spot any glaring errors by later today, I will merge this branch.

Thanks for your help improving elexon 😊