Calling get_historic_weather error

[scheung38]

Running sample shows

data_historic = ow.get_historic_weather(station_id, start_date, end_date) returning error:

ow = OpenWeatherMapRequests(api_key='My API-key', cache_name='cache-openweathermap', expire_after=5*60)
(lon, lat) = (2.34880, 48.853409)  # Paris

# Display current weather data
data = ow.get_weather(lon=lon, lat=lat)
print "Current Paris, FR weather data"
print data # (objects returned)

# Get 10 nearest stations from coordinate
stations = ow.find_stations_near(lon=lon, lat=lat, cnt=10)
print "Top 10 nearest stations"
print stations # Top 10 stations returned...

# Get station_id of nearest station
station_id = stations.iloc[0]['station.id']
print "Nearest Station"
print station_id # 137218

start_date = datetime.datetime(2015, 7, 19)
end_date = datetime.datetime(2015, 8, 19)
print start_date # 2015-07-19 00:00:00
print end_date # 2015-08-19 00:00:00

# Get historic weather from start to end date
data_historic = ow.get_historic_weather(station_id, start_date, end_date) # error here???
# print data_historic

As you can see all input arguments to get_historic_weather are returned, but calling it I get:

   raise ValueError('All objects passed were None')
ValueError: All objects passed were None

Anyone?

[scls19fr]

please show error

[scheung38]

Hi you answered in another post where there is empty station_id, etc but here I clearly have printed valid objects before being passed to get_historic_weather function and it is here the scenario returning this error. Thanks.

[scls19fr]

station_id is not empty. That's historic data from OpenWeatherMap which is empty and I can't do anything about it.

[scheung38]

Hi, a bit confused, you mentioned OpenWeatherMapRequests is empty? But your usage example uses this as object to get historical data?

[scls19fr]

http://api.openweathermap.org/data/2.5/history/station?type=hour&start=1436646230&end=1437510229&id=5530&appid=YOURAPPID

{
"message": "",
"cod": "200",
"type": "hour",
"station_id": 5530,
"calctime": 0.0013,
"cnt": 0,
"list": []
}

"list" is an empty list

[scheung38]

If I do it this way

import pandas as pd
json_str = 'http://api.openweathermap.org/data/2.5/history/city?q=Paris,FR&type=hour&start=1437264000&end=1439942400'
paris = pd.read_json(json_str, typ='series')

[scls19fr]

Ok, I wasn't using /history/city endpoint

If you want to do it yourself, you might do

import datetime
import requests
import pandas as pd
from openweathermap_requests import historic_weather_to_df, datetime_to_timestamp
base_url = 'http://api.openweathermap.org/data/2.5'
endpoint = '/history/city'
dt_start = datetime.datetime(2015, 7, 19, 0, 0)
dt_end = datetime.datetime(2015, 8, 19, 0, 0)
dt_start = datetime_to_timestamp(dt_start)
dt_end = datetime_to_timestamp(dt_end)
params = {
    'q': 'Paris,FR',
    'type': 'hour',
    'start': dt_start,
    'end': dt_end,
}
url = base_url + endpoint
response = requests.get(url, params=params)
status_code = response.status_code
if status_code == 200:
    data = response.json()
else:
    raise(Exception("HTTP status code is %d" % status_code))
#df = pd.DataFrame(data['list'])
#from pandas.io.json import json_normalize
#df = json_normalize(data['list'])
df = historic_weather_to_df(data)

[scheung38]

Would be great if you could show how to visualise Plotly with tabs showing various data such as temp, humidity etc as in their OpenWeather site.

[scheung38]

Does your package also show to how to test reading in and tests if visualising data correctly? If not, any pointers to start unit testing would be fantastic.

[scls19fr]

I don't use plotly but according https://plot.ly/python/time-series/ you should do

import plotly.plotly as py
from plotly.graph_objs import *
x = df.index.to_datetime()
y = df['main.temp'].values
data = Data([
    Scatter(
        x=x,
        y=y
    )
])
plot_url = py.plot(data, filename='python-datetime')

I don't understand your last comment.

[scls19fr]

https://plot.ly/~SebastienCelles/3

[scheung38]

Oh, last comment was looking to try and do simple unittest on retrieved data, and if visualising is correct as expected

[scheung38]

Tried to replicate but got exactly the same shape of graph but factor of 18x (16.05 deg as opposed to 289.2) which is clearly wrong for temperature, but that's the returned JSON, so just wondering how you got the final correct temperature data. Even if in Fahrenheit it would still be incorrect?

Also, getting the below warning:

data = Data([
    Scatter( <--- Unresolved reference Scatter?
        x=x,
        y=y
    )
])

http://api.openweathermap.org/data/2.5/history/city?q=Paris,FR&type=hour&start=1437264000&end=1439942400

response:

{
    "message": "",
    "cod": "200",
    "city_id": 2988507,
    "calctime": 0.6498,
    "cnt": 115,
    "list": [
        {
            "main": {
                "temp": 289.2,
                "pressure": 1013,
                "humidity": 88,
                "temp_min": 288.15,
                "temp_max": 290.15
            },

https://plot.ly/~SebastianCheung/33

[scls19fr]

That's a Kelvin/Celcius conversion error.

You need to install an updated version of https://github.com/scls19fr/openweathermap_requests

Get latest version using Git

$ git clone https://github.com/scls19fr/openweathermap_requests.git
$ cd openweathermap_requests
$ python setup.py install

or

$ pip install git+https://github.com/scls19fr/openweathermap_requests.git

[scheung38]

Thanks the update seems to work, but did noticed before it was version 0.05 and it is still 0.05, but works now anyway. Thanks.

[scls19fr]

I haven't update version number because I haven't push to pypi

[scheung38]

Any idea how to plot humidity, etc either as multi plots or switchable tabs ?

[scls19fr]

You can try

trace0 = Scatter(
    x = df.index.to_datetime(),
    y = df['main.temp'].values,
    mode='lines+markers',
    name = 'temp'
)

trace1 = Scatter(
    x = df.index.to_datetime(),
    y = df['main.humidity'].values,
    mode='lines+markers',
    name = 'humidity'
)

data = Data([trace0, trace1])

plot_url = py.plot(data, filename='python-datetime')

but I don't know how to (or if we can) have tabs with plotly

[scheung38]

Tried to plot weather icon, or weather main which are of type string it seems but both are returning KeyError:

trace5 = Scatter(
        x=df.index.to_datetime(),
        y=df['weather.icon'].str,
        name='weather_icon'
    )
trace6 = Scatter(
        x=df.index.to_datetime(),
        y=df['weather.main'].str,
        name='weather_main'
    )

[scls19fr]

In [3]: df.columns
Out[3]:
Index(['clouds.all', 'main.humidity', 'main.pressure', 'main.temp',
       'main.temp_max', 'main.temp_min', 'rain.1h', 'weather', 'wind.deg',
       'wind.gust', 'wind.speed', 'wind.var_beg', 'wind.var_end'],
      dtype='object')

there is no weather.main in DataFrame columns

moreover why using this .str attribute ?

df['wind.speed'].str

raises

AttributeError: Can only use .str accessor with string values, which use np.object_ dtype in pandas

[scheung38]

Because weather.icon and weather.main are if type string. So tried plotting them using .str but not so sure how

[scheung38]

If refactoring into a function, say myPlot() then unittesting, not sure how to test it as it would return unicode

plot_url returns u'https://plot.ly/~SebastianCheung/79' 

and each time a new number would be recreated:

import unittest

class MyTestCase(unittest.TestCase):
    def test_something(self):
        self.assertEqual(myPlot, ???)

if __name__ == '__main__':
    unittest.main()