Import data request meter data

[macintoshpie]

Open Tech request

[macintoshpie]

To export meter data from ESPM, you must manage the property yourself or have the property shared with you. If you create a data request that includes monthly meter data, SEED is unable to import this from a spreadsheet currently b/c it's structured differently.

Managed export columns under 'Meter Entries' sheet: Property Name	Portfolio Manager ID	Portfolio Manager Meter ID	Meter Name	Meter Type	Meter Consumption ID	Start Date	End Date	Delivery Date	Usage/Quantity	Usage Units	Cost ($)	Estimation?	Demand (kW)	Demand Cost ($)	Last Modified Date	Last Modified By

Data request columns under 'Monthly Usage' sheet:

Property Id	Property Name	Parent Property Id	Parent Property Name	Month	Electricity Use  (GJ)	Natural Gas Use  (GJ)

Notable differences:

• for data request, instead of a start/end time, each row is a month of a year. I assume we'd just use the first and last day of the month.

• for data request, instead of separate rows for different energy sources, there's a column for every energy type requested (important to note that the only monthly data you can request is Electricity and / or Natural Gas)

  

• for data request, there's no meter ID. Is our best option to just use PM Property ID? How problematic could that be down the road? This looks like this is used for Meter.source_id.

• for data request, there's no meter type. Should we assume Electric - Grid and Natural Gas?

• for data request, we must parse units from the column name rather than from its own column value

Limitations:

• how do we handle a property that has multiple meters of the same resource type (e.g. electricity)? Technically these meters should have a different source id, but that's not provided here. Is this a valid concern, does this happen in real life?
• for data request, we will only be able to import electricity and natural gas, not other fuel types
• for data request, we cannot have fuel cost

@adrian-lara @nllong Can you review the above and let me know what you think? Specifically how to handle the missing columns. It seems very doable to allow importing meter data from data request spreadsheets, but it is some effort.

[adrian-lara]

As mentioned over the call, the previous work related to this is this issue and the corresponding PR.

Given this was built before fully understanding the limitations of this route, this is just for reference and may not ultimately be useful going forward.

[macintoshpie]

After discussion, we identified some problems that make importing this type of file undesirable. When you export meters through the documented flow the spreadsheet includes "Portfolio Manager Meter ID" and "Meter Type", which the data request is missing.

"Meter Type" is used to determine the "source", which can be "Electricity - Grid", "Electricity - Solar" or "Electricity - Wind". If you knew what the source was, either added to the spreadsheet or selected when importing the spreadsheet, then we could handle this. But note that your selection would apply to all readings for that property (there's only one Electricity column in your data), which might not be accurate (e.g. a property might have both grid and solar).

Portfolio Manager Meter ID is used to discriminate multiple meters for the same property. E.g. you could have multiple "Electricity - Grid" meters for a single property in ESPM. From what we can tell, ESPM aggregates all of the meters into monthly readings when you use a data request template. SEED doesn't currently have an aggregate representation of meters, but it is possible that we could add this.

For these reasons we are recommending not implementing this until it's certain that this is our only option (ie that properties cannot be shared rather than responding to a data request)

[macintoshpie]

It appears that data sharing is not an option, and it is OK that only electricity and natural gas can be imported. In addition it appears to be ok that they are aggregated. To account for the above limitations I propose the following:

• Create a new import flow for importing meters from a data request
• Create a new source type for meters in SEED: "Electricity - Unknown" and "Natural Gas - Unknown"
• Create the meter readings with these new source types
• Create the meter readings without source_id since there is none

To summarize, the only meaningful difference between these meters and the ones we currently support is that they are a different type (aggregated) and that they don't have a source_id.

Remaining questions:

• If a property has an aggregate meter, can they also have individual meters?
  • yes, that should be no problem
• Are we supposed to validate at time t: sum(individual meters_t) <= aggregate meter_t?
  • I would say no for now
• if a user imports the same meter data more than once, does it create new or update meter?
  • update (like current meter import)