search

alan-turing-institute / CROP

CROP is a Research Observation Platform
MIT License
25 stars 4 forks source link

How to get API endpoint for weather data #106

Closed myyong closed 3 years ago

myyong commented 3 years ago

Purpose: This issue records progress on getting additional data into the database. In this case, we are looking for weather data. @meljsingh makes a strong case for pulling data from the Zensie platform:

On the other hand, on the zensie platform, GU pays for temperature, relative humidity, air pressure data from meteomatics weather station, as in picture attached. We could access this in a similar way to the other Zensie data. Furthermore, the GES (Rebecca’s model) also needs external relative humidity, so I think it would be more suited!

Therefore we are looking for the API endpoint to allow us to pull weather data.

Background: Currently we pull data from Zensie using the following API call:

https://api.30mhz.com/api/stats/check/<api_key>/from/2021-04-17T04:59:31Z/until/2021-04-17T05:59:31Z?statisticType=averages&intervalSize=5m

which returns (a subset)

 {
    "1618635600000": {
        "bb4ae4c5_8604_11ea_9717_ab99b5272044.temperature": 17.899999618530273,
        "bb4ae4c5_8604_11ea_9717_ab99b5272044.humidity": 55.0
    },
    ...
}

Success Criteria: The issue is closed when we have an API endpoint to return weather data

myyong commented 3 years ago

References 1) Zensie's landing documentation for API use 2) Zensie API testing site

myyong commented 3 years ago

To get a list of sensors available to us, I can use https://api.30mhz.com/api/sensor-type which returns a large JSON file

The sections pertaining to weather data in the file are probably:

{
        "typeId": "i_rweatherstation",
        "name": "Ridder weather station (imported)",
        "createdAt": null,
        "description": "Ridder weather station (from external source)",
        "icon": "<i class=\"material-icons\">wb_sunny</i>",
        "color": null,
        "chartColor": null,
        "metrics": [
            "temperature",
            "wind",
            "angle",
            "radiation",
            "energy_in_area",
            "light",
            "parsum",
            "radiation",
            "rainintensity",
            "humidity",
            "humidity-deficit",
            "temperature",
            "specific-humidity",
            "enthalpy",
            "barometricpressure",
            "boolean_number"
        ],
        "decimals": 1,
        "jsonKeys": [
            "i_rweatherstation.temp",
            "i_rweatherstation.windsp",
            "i_rweatherstation.winddir",
            "i_rweatherstation.rad",
            "i_rweatherstation.radsum",
            "i_rweatherstation.par",
            "i_rweatherstation.parsum",
            "i_rweatherstation.outrad",
            "i_rweatherstation.rainint",
            "i_rweatherstation.hum",
            "i_rweatherstation.hd",
            "i_rweatherstation.dew",
            "i_rweatherstation.abshum",
            "i_rweatherstation.ent",
            "i_rweatherstation.pressure",
            "i_rweatherstation.rain"
        ],
        "jsonLabels": [
            "Temperature",
            "Wind speed",
            "Wind direction",
            "Radiation",
            "Radiation sum",
            "PAR",
            "PAR sum",
            "Outgoing radiation",
            "Rain intensity",
            "Relative humidity",
            "Humidity deficit",
            "Dew point temperature",
            "Absolute humidity",
            "Enthalpy",
            "Barometric pressure",
            "Rain"
        ],
        "queryParametersRequired": null,
        "queryParameters": null,
        "hasVoltageInformation": null,
        "batteryVoltageChargedEnough": null,
        "batteryVoltageWarning": null,
        "batteryVoltageCritical": null,
        "isBillable": true,
        "rangeMin": null,
        "rangeMax": null,
        "notificationTemplates": null,
        "external": true,
        "isForManualInput": null,
        "dataTypes": [
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "long"
        ],
        "enums": null,
        "radioId": null,
        "productionStatus": null,
        "canSleep": null,
        "metadata": null
    },

and 

{
        "typeId": "i_weathers",
        "name": "Weather Data",
        "createdAt": null,
        "description": "Weather Data",
        "color": null,
        "chartColor": null,
        "metrics": [
            "temperature",
            "wind",
            "percentage",
            "rainfall",
            "radiation",
            "humidity",
            "angle_wind",
            "barometricpressure"
        ],
        "decimals": 1,
        "jsonKeys": [
            "i_weathers.temperature",
            "i_weathers.wind_speed",
            "i_weathers.rain_probability",
            "i_weathers.rain",
            "i_weathers.radiation",
            "i_weathers.relative_humidity",
            "i_weathers.wind_direction",
            "i_weathers.air_pressure"
        ],
        "jsonLabels": [
            "Temperature",
            "Wind speed",
            "Rain probability",
            "Rain",
            "Global radiation ",
            "Relative humidity",
            "Wind direction",
            "Air pressure"
        ],
        "queryParametersRequired": null,
        "queryParameters": null,
        "hasVoltageInformation": null,
        "batteryVoltageChargedEnough": null,
        "batteryVoltageWarning": null,
        "batteryVoltageCritical": null,
        "isBillable": true,
        "rangeMin": null,
        "rangeMax": null,
        "notificationTemplates": null,
        "external": true,
        "isForManualInput": null,
        "dataTypes": [
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double",
            "double"
        ],
        "enums": [],
        "radioId": null,
        "productionStatus": null,
        "canSleep": null,
        "metadata": null
    },

What are the differences? Using https://api.30mhz.com/api/sensor-type/ only returns the associated sections above.

myyong commented 3 years ago

https://api.30mhz.com/api/check/organization/growingundergound returns the list of sensors as below (subset ):

{
        "name": "Sensor 0013A200419F7BC5",
        "url": "http://localhost:8080/zensie/0013A200419F7BC5/zensie_router",
        "locations": [
            "growingunderground"
        ],
        "frequency": 60,
        "healthyThreshold": 7,
        "maxReminders": 5,
        "method": "get",
        "remindEvery": 300,
        "timezone": "Europe/London",
        "timezoneOffset": 60,
        "collectJson": true,
        "statusCodes": [
            200
        ],
        "timeout": 10,
        "unhealthyThreshold": 3,
        "allowRedirects": false,
        "enabled": true,
        "status": "ACTIVE",
        "decimalsPerField": {
            "voltage": 1
        },
        "maxDecimals": 1,
        "stream": false,
        "locationThreshold": 1,
        "verify": true,
        "sensorType": "zensie_router",
        "email": "factory@30mhz.com",
        "organizationId": "growingundergound",
        "checkId": "fdbf0904-0b38-488f-b79f-a13dc4500b10",
        "createdAt": "2020-03-13T14:04Z",
        "state": "up",
        "hasNotifications": false,
        "owner": "factory@30mhz.com"
    },

We should be able to use something like https://api.30mhz.com/api/stats/check/{{checkID}}?intervalSize=60m to get data from this check. But we are currently using a checkID=i-9c2f5171-8600-11ea-9717-bfeb7a197176. Where does it come from?

myyong commented 3 years ago

The checkID for Meteomatics Station Growing Underground is i-b9bf5432-9436-11ea-b286-79ffe41fb933.

I got it from the web browser as suggested in the documentation . There is a screenshot here.

Therefore https://api.30mhz.com/api/stats/check/i-b9bf5432-9436-11ea-b286-79ffe41fb933/from/2021-04-17T04:59:31Z/until/2021-04-17T05:59:31Z?statisticType=averages&intervalSize=60m produces:

{
    "1618635600000": {
        "i_weathers.temperature": 1.2000000476837158,
        "i_weathers.rain_probability": 1.0,
        "i_weathers.relative_humidity": 88.5,
        "i_weathers.rain": 0.0,
        "i_weathers.radiation": 0.0,
        "i_weathers.wind_speed": 2.5,
        "i_weathers.air_pressure": 1028.5,
        "i_weathers.wind_direction": 38.400001525878906
    }
}