RFC: Design for system to gather and report live charging station status worldwide

webprofusion-chrisc commented 8 years ago

We are now at a point were we need to be able to provide live status information for charging equipment via Open Charge Map.

Primary Requirements:

Provide near-real time status (availability/status) for any charging equipment where the network shares that information with us. The main purpose of this is for drivers to know what is available, and where.
Provide historical status information (granularity and retention period to be discussed). The main purpose of this is for reliability analytics and availability prediction.

Data may be sourced in a variety of manners including real-time polling (on demand), batch updates (pull and push), streaming status APIs. We will require that all the data we consume is either Open Data or free from copyright claims.

Traditional implementations of equipment monitoring use a combination of in-memory or temporary database and 'historian' systems for archiving and reporting time-series values.

webprofusion-chrisc commented 8 years ago

Which Providers currently offer status information as Open Data?

http://www.nobil.no/ [realtime, streaming]

isj4 commented 8 years ago

I'm about to develop real-time status information from a system. I'm thinking of http/https put with json payload would do the trick (at least for us). Would you be interesting in helping with hammering out the details so it will be easily usable by OCM?

webprofusion-chrisc commented 8 years ago

@isj4 sounds interesting! Our requirement is to be able to aggregate status information from multiple sources (usually JSON feeds) and possibly also via a reporting agent which network operators install (probably using OCCP) then feeds changes to us. We then need to be able to report status for a given equipment item via our public API (to apps/websites etc using our service). We probably also want to keep recent status (change) history per device.

isj4 commented 8 years ago

Ok! Background: we specialize in energy-efficiency (mostly for hotels) by installing and managing intelligent thermostats, outlets, lights, etc. We also handles DRs (demand-response (=fancy word for when utilities send out a requests to lower electrical power usage)), and also do some peak load shaving (reduces hotel's power bills). One hotel is currently installing a number of EVSE, which we will know the status of and also be able to control. The goal is that the customer (the hotel) can simply click "Publish as Open Data" and fill out a few pieces of information (eg. availability, price, directions) then we will take care of pushing that information out to OCM (and probably others).

So what I'm thinking of is to simply do a http/https PUT to a fixed prior-agreed-upon URL with a JSON payload containing location and station information. I've currently come up with this example:

{
    "id": "sPd-5563-sRd8-48502814730100/gh_2",
    "name": "Slot 201",
    "state": "free",
    "last_state_change" : "2016-06-09T22:12:43Z",
    "availability": "guests",
    "cost": "gratis",
    "outlets" : [
        "SAE J1772",
        "Tesla Roadster",
        "NEMA 14-50",
        "Zogg's special plug Z-12"
    ],
    "rating_volts": 12.0,
    "rating_amps": 62,
    "current_max_charging_level": 50,
    "data_freshness": "2016-06-09T22:17:26Z",
    "location" : {
        "id": "qPd-5563-sRd8-48502814730100/Jh_2",
        "name": "Dancing Badger Hotel, 5th floor",
        "latitude": 37.2953577,
        "longitude": -115.817374,
        "address": "Homey Airport\nEdwards Air Force Base",
        "directions": "Turn left after the 4th circular aircraft,\nthen follow vertical light beam."
    }
}

The PUT will be done to a fixed URL regardless of which station it is because we don't know about the recipient's data model (and don't care). Thing things that have to be agreed upon is:

the URL
HTTP basicauth username+password
location.id (except when the recipient fully trusts us and allows us to create locations willy-nilly)

The PUT is done when the state of the station changes. And periodically as a sort of keep-alive.

The only historical information is last_state_change which may be missing if the hotel considers it a privacy issue.

The location is redundant across stations but I'd like to keep the protocol simple and just have a single type of message.

I'm not sure how to signal that a station is gone/deleted.

What do you think?

webprofusion-chrisc commented 8 years ago

Looks good but it could probably be split into two things, one is charging station details (changes infrequently) and the other is current status (changes regularly), Where you are writing to some else's API you would generally follow a format they specify, rather than designing one that's for your system only. It's unlikely you will get multiple providers to code parsers for your own format.

For OCM we will be shifting to using an authorization step (POST username/pwd, receive a jwt token), you will then specify an Authorization: Bearer header in your subsequent PUT/POST until that jwt expires or until the server wants to authenticate again. As the server will be deciding how to handle the input, this is probably a POST rather than a PUT (to a discrete entity).

data_freshness sounds funny, maybe date_modified?

A station may have multiple ports which can service one or more vehicles at a time or may work exclusively (if your using one then they are all considered in use).

For our "live" status api we intend to separate status from equipment specifications, so you would only need to provide your equipment ID, a network operator name/id and status information. This could be in use/available/reserved/not operational etc.

OCM has an API for accepting charging station submissions but we don't yet have an API dedicated to status updates (we need it soon).

webprofusion-chrisc commented 8 years ago

We would actually like to write a free tool which the Network operator runs, which would gather equipment status and push it back to us when changes occur, saving them the trouble of individually developing such tools. This would most likely be a service with a front end for easier configuration/status checks.

isj4 commented 8 years ago

The location and the station information can easily be split into two distinct messages. No problem.

Authentication: if a separate authentication step makes it easier for you, sure. As long as it is not something unnecessarily complex.

So something like 4 URLs would be ideal for OCM? :

/.../authenticate
/.../location
/.../station
/.../station_status

Multiple ports per station: Yes. An EVSE can have multiple plugs, eg. "SAE J1772" and "Tesla Roadster", Some of the larger EVSEs may also be able to serve multiple vehicles at the same time, but that is irrelevant to the user. What might not have been clear from my example was that the whole json entity was a single station/charging-point/spot/thingamajig

webprofusion-chrisc commented 8 years ago

Yes, more or less. Currently our API is a post to /v2/poi which can be either a new or updated site+equipment.

webprofusion-chrisc commented 8 years ago

Having thought about this recently we currently do things a little backwards by being site (location) centric and we currently mix core facts with subjective info (like access comments etc). We could really consider each physical station as one or more connectors, with a position, usage type, status and associated network operators. The station can have an overall status (and history) and the connectors can have an individual status (and history). The station is then associated with a site (which has an approximate position and community supplied extended info, this pulls it together mainly for the benefit of the end user).

Once the core facts about a station have been established we could then accept status info.

isj4 commented 8 years ago

I had a quick look at OCCP (no useful stuff) and OCHP (not really useful).

How about this:

Purpose: The purpose of the protocol is to allow charging station owners to publish information on locations and stations and real-time state of stations.

Terminology: site/location :: A physical location where zero or more stations exist station/charging-point :: a sub-location where zero or more outlets/plugs exist plug/outlet :: One of the standard plug type technical names, eg. "SAE J1772"

The endpoints share a common URL prefix, eg https://api.example.com/v17/

Endpoint: /authenticate

Used for obtaining an Authorization token for accessing other endpoints Example POST:

{
    "username" : "zogg",
    "password": "1234"
}

Example response:

{
    "authorization": "458593-39407-a743b3"
}

Endpoint: /location

Used for creating or updating a location Example POST:

{
    "id": "qPd-5563-sRd8-48502814730100/Jh_2q",
    "name": "Dancing Badger Hotel, 5th floor",
    "latitude": 37.2953577,
    "longitude": -115.817374,
    "address": "Homey Airport\nEdwards Air Force Base",
    "directions": "Turn left after the 4th circular aircraft,\nthen follow vertical light beam."
}

Endpoint: /station

used for creating or updating a station

{
    "id": "sPd-5563-sRd8-48502814730100/gh_2",
    "name": "Slot 201",
    "location_id": "qPd-5563-sRd8-48502814730100/Jh_2q",
    "state": "free",
    "last_state_change" : "2016-06-09T22:12:43Z",
    "availability": "guests",
    "cost": "gratis",
    "outlets" : [
        "SAE J1772",
        "Tesla Roadster",
        "NEMA 14-50",
        "Zogg's special plug Z-12"
    ],
    "rating_volts": 12.0,
    "rating_amps": 62,
    "current_max_charging_level": 50,
    "date_modified": "2016-06-09T22:17:26Z",
}

Endpoint: /station_state

Used for updating the station state, availability and/or current_max_charging_level Example POST:

{
    "id": "sPd-5563-sRd8-48502814730100/gh_2",
    "state": "free",
    "last_state_change" : "2016-06-09T22:12:43Z",
    "availability": "guests",
    "current_max_charging_level": 50,
}

isj4 commented 8 years ago

Was my proposal completely bonkers?

webprofusion-chrisc commented 8 years ago

HI Ivan, sorry I missed this notification. No it's not bonkers at all! I'd need to re-work/re-word it more to fit into our existing API model/fields but the general idea is indeed good.

isj4 commented 8 years ago

Ok, then I have some more details regarding the fields:

station.state ::

free
out_of_service
connected
charging Note: an owner may not wish to distinguish connected vs. charging due to privacy reasons.

station.availability ::

public (available to the general public)
guests (available to visitors, guests, customers)
private

station.cost ::

gratis
limited_gratis (certain restrictions may apply, eg. first 30 minutes are gratis, or non-visitors has to pay)
fee

webprofusion-chrisc commented 8 years ago

Thanks Ivan, so you can see what I'm trying to align with, here is our current API docs (quite limited): http://openchargemap.org/site/develop/api

An example POI Model with equipment, comments etc looks like:

http://api.openchargemap.io/v2/poi?chargepointid=9797

The reference data codes for our connectors, status types etc are here:http://api.openchargemap.io/v2/referencedata

We are not quite starting from scratch as we need to minimize the level of changes for our API users as they go up to the next version, which is why we need to only change what's missing etc.

webprofusion-chrisc commented 8 years ago

On a related topic, here is a current proposed standard for charging equipment information exchange:

http://en.nklnederland.nl/uploads/files/OCPI_2.1.pdf

isj4 commented 8 years ago

Sorry - I didn't get a notification of your message. I'll read the specs.

isj4 commented 8 years ago

I've had time to read the nice links you posted. I'm flexible as long as I can achieve the desired outcome: to make it easy for our clients to create and update locations, chargingpoints and status. But I'm not sure how to design an API that makes life easy for you.

OCPI looks reasonable but I'm not sure how it maps to the object model in OCM

openchargemap / ocm-system