search

flair-systems / flair-api-client-py

Python Client for Flair API
Apache License 2.0
15 stars 5 forks source link

API Keeps failing to retrieve data #2

Open NickWaterton opened 5 years ago

NickWaterton commented 5 years ago

This is the error that I have been getting recently:

****** Program Started ********
Debug Mode
reading info from config file ./config.ini
reading house: xxxx
Authenticating
Authenticated for 3600s
Authentication expires in: 3599s
got data for 8 vents
Vent: Nick's Office-5384, Latest Reading: {
  "Pressure": 100.83,
  "RSSI": -68,
  "Temperature": 53.59,
  "bat_percent": "100",
  "battery": 3.06,
  "date": "2019-03-19T16:56:59.097290-04:00",
  "online": true,
  "percent_open": 1,
  "set_by": "Algo"
}
Vent: Hallway-9e65, Latest Reading: {
  "Pressure": 100.77,
  "RSSI": -60,
  "Temperature": 56.63,
  "bat_percent": "100",
  "battery": 3.55,
  "date": "2019-03-19T16:56:48.326764-04:00",
  "online": true,
  "percent_open": 99,
  "set_by": "Algo"
}
Vent: Family Room-7cf9, Latest Reading: {
  "online": false
}
Vent: Washroom-850c, Latest Reading: {
  "Pressure": 100.83,
  "RSSI": -60,
  "Temperature": 51.22,
  "bat_percent": "100",
  "battery": 3.54,
  "date": "2019-03-19T16:56:48.325377-04:00",
  "online": true,
  "percent_open": 1,
  "set_by": "Algo"
}
Vent: Family Room-9b19, Latest Reading: {
  "Pressure": 100.87,
  "RSSI": -68,
  "Temperature": 56.15,
  "bat_percent": "100",
  "battery": 3.53,
  "date": "2019-03-19T16:56:59.098442-04:00",
  "online": true,
  "percent_open": 0,
  "set_by": "Algo"
}
Vent: Master Bedroom-d7df, Latest Reading: {
  "Pressure": 100.78,
  "RSSI": -64,
  "Temperature": 54.18,
  "bat_percent": "100",
  "battery": 3.07,
  "date": "2019-03-19T16:56:59.095481-04:00",
  "online": true,
  "percent_open": 99,
  "set_by": "Algo"
}
Vent: Dining Room-744e, Latest Reading: {
  "online": false
}
Vent: Basement-5da4, Latest Reading: {
  "Pressure": 100.76,
  "RSSI": -52,
  "Temperature": 56.48,
  "bat_percent": "100",
  "battery": 3.55,
  "date": "2019-03-19T16:56:48.328508-04:00",
  "online": true,
  "percent_open": 100,
  "set_by": "Algo"
}
got data for 7 pucks
Puck: Francesca's Bedroom-bbd1, Latest Reading: {
  "Humidity": 44.0,
  "Pressure": 100.66,
  "RSSI": -82,
  "Temperature": 20.09,
  "active": true,
  "bat_percent": "59",
  "battery": 2.76,
  "date": "2019-03-19T16:56:59.092375-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 249,
  "occupied": false,
  "online": true,
  "room_name": "Francesca's Bedroom",
  "set_by": "Algo",
  "set_point_c": 18.0,
  "set_point_manual": false,
  "set_temp": 18.0
}
Puck: Hallway-d02f, Latest Reading: {
  "Humidity": 40.0,
  "Pressure": 100.74,
  "RSSI": -85,
  "Temperature": 22.7,
  "active": true,
  "bat_percent": "63",
  "battery": 2.78,
  "date": "2019-03-19T16:56:48.329848-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 79,
  "occupied": true,
  "online": true,
  "room_name": "Hallway",
  "set_by": "Algo",
  "set_point_c": 22.0,
  "set_point_manual": false,
  "set_temp": 22.0
}
Puck: Washroom-ec75, Latest Reading: {
  "Humidity": 42.0,
  "Pressure": 100.7,
  "RSSI": -65,
  "Temperature": 24.05,
  "active": true,
  "bat_percent": "86",
  "battery": 2.92,
  "date": "2019-03-19T16:57:13.238292-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 6,
  "occupied": true,
  "online": true,
  "room_name": "Washroom",
  "set_by": "Algo",
  "set_point_c": 22.0,
  "set_point_manual": false,
  "set_temp": 22.0
}
Puck: Nick's Office-a762, Latest Reading: {
  "Humidity": 41.0,
  "Pressure": 100.71,
  "RSSI": -79,
  "Temperature": 26.52,
  "active": true,
  "bat_percent": "91",
  "battery": 2.95,
  "date": "2019-03-19T16:56:59.089290-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 19,
  "occupied": true,
  "online": true,
  "room_name": "Nick's Office",
  "set_by": "Algo",
  "set_point_c": 22.0,
  "set_point_manual": false,
  "set_temp": 22.0
}
Puck: Basement-6f36, Latest Reading: {
  "Humidity": 46.0,
  "Pressure": 100.79,
  "RSSI": 0,
  "Temperature": 23.0,
  "active": true,
  "bat_percent": "100",
  "battery": 3.41,
  "date": "2019-03-19T16:57:13.240106-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 1,
  "occupied": true,
  "online": true,
  "room_name": "Basement",
  "set_by": "Algo",
  "set_point_c": 22.0,
  "set_point_manual": false,
  "set_temp": 22.0
}
Puck: Master Bedroom-0b07, Latest Reading: {
  "Humidity": 45.0,
  "Pressure": 100.72,
  "RSSI": 0,
  "Temperature": 23.57,
  "active": true,
  "bat_percent": "100",
  "battery": 3.38,
  "date": "2019-03-19T16:56:59.093629-04:00",
  "hold_reason": null,
  "hold_until": null,
  "light": 98,
  "occupied": true,
  "online": true,
  "room_name": "Master Bedroom",
  "set_by": "Algo",
  "set_point_c": 22.0,
  "set_point_manual": false,
  "set_temp": 22.0
}
Traceback (most recent call last):
  File "./vents_bridge.py", line 1197, in <module>
    main()
  File "./vents_bridge.py", line 1172, in main
    PublishPuck_data(structure, mqttc)
  File "./vents_bridge.py", line 818, in PublishPuck_data
    sensor_readings = puck.get_rel('sensor-readings')
  File "build/bdist.linux-x86_64/egg/flair_api/client.py", line 131, in get_rel
  File "build/bdist.linux-x86_64/egg/flair_api/client.py", line 50, in get
  File "build/bdist.linux-x86_64/egg/flair_api/client.py", line 303, in get_url
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 658, in send
    r.content
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 823, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 748, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

Sometimes i only get one or two readings before the error.

Note: There is a long pause (like a timeout) between the last reading and the error. Normally the readings come in one after another, and the cycle is repeated every minute.

For completeness, this is what you get under python 3.6:

****** Program Started ********
reading info from config file ./config.ini
reading house: 1234
Authenticating
Authenticated for 3600s
Authentication expires in: 3599s
got data for 8 vents
Vent: Nick's Office-5384, Latest Reading: {
  "Pressure": 99.04,
  "RSSI": -67,
  "Temperature": 22.13,
  "bat_percent": "100",
  "battery": 3.06,
  "date": "2019-03-21T15:49:48.330632-04:00",
  "online": true,
  "percent_open": 1,
  "set_by": "Algo"
}
Vent: Hallway-9e65, Latest Reading: {
  "Pressure": 99.01,
  "RSSI": -56,
  "Temperature": 23.58,
  "bat_percent": "100",
  "battery": 3.51,
  "date": "2019-03-21T15:49:42.166232-04:00",
  "online": true,
  "percent_open": 99,
  "set_by": "Algo"
}
Vent: Family Room-7cf9, Latest Reading: {
  "Pressure": 99.06,
  "RSSI": -70,
  "Temperature": 23.3,
  "bat_percent": "100",
  "battery": 3.5,
  "date": "2019-03-21T15:49:48.335045-04:00",
  "online": true,
  "percent_open": 0,
  "set_by": "Algo"
}
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 543, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 302, in _error_catcher
    yield
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 598, in read_chunked
    self._update_chunk_length()
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 547, in _update_chunk_length
    raise httplib.IncompleteRead(line)
http.client.IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/models.py", line 745, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 432, in stream
    for line in self.read_chunked(amt, decode_content=decode_content):
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 626, in read_chunked
    self._original_response.close()
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3/dist-packages/urllib3/response.py", line 320, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./vents_bridge.py", line 1201, in <module>
    main()
  File "./vents_bridge.py", line 1175, in main
    PublishVent_data(structure, mqttc)
  File "./vents_bridge.py", line 777, in PublishVent_data
    sensor_readings = vent.get_rel('sensor-readings')
  File "/usr/local/lib/python3.6/dist-packages/flair_client-0.2.0-py3.6.egg/flair_api/client.py", line 131, in get_rel
  File "/usr/local/lib/python3.6/dist-packages/flair_client-0.2.0-py3.6.egg/flair_api/client.py", line 50, in get
  File "/usr/local/lib/python3.6/dist-packages/flair_client-0.2.0-py3.6.egg/flair_api/client.py", line 302, in get_url
  File "/usr/lib/python3/dist-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 670, in send
    r.content
  File "/usr/lib/python3/dist-packages/requests/models.py", line 823, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 748, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

My source code is here https://github.com/NickWaterton/Flair-Vents-MQTT-Interface

NickWaterton commented 5 years ago

I'm fairly sure this https://github.com/urllib3/urllib3/issues/1516 is the issue.

I have implemented a workaround by adding , timeout=5.0 to each of your api requests.get calls, and trapping the timeout in my code. This seem to work for now.

edpaget commented 5 years ago

Thanks nick I will get this fixed on the server side as well.

edpaget commented 5 years ago

@NickWaterton I've made some further API changes today. Can you let me know if you're still seeing this issue?

NickWaterton commented 5 years ago

Ed,

Are these changes to the API, or the server? The server is the current issue. Using the 5 second timeout and my retry mechanism, I see the error every 5 minutes or so, which is a lot less than it used to be. Sometimes it can go 20 minutes or more with no error.

Nick Waterton P.Eng. National Support Leader - Nuclear Medicine, PET and RP GE Healthcare

M+1 416 859 8545 E nick.waterton@med.ge.commailto:nick.waterton@med.ge.com

2300 Meadowvale Boulevard Mississauga, ON L5N 5P9 Canada

GE imagination at work

From: Edward Paget notifications@github.com Sent: Tuesday, March 26, 2019 3:51 PM To: flair-systems/flair-api-client-py flair-api-client-py@noreply.github.com Cc: Waterton, Nick (GE Healthcare) Nick.Waterton@med.ge.com; Mention mention@noreply.github.com Subject: EXT: Re: [flair-systems/flair-api-client-py] API Keeps failing to retrieve data (#2)

@NickWatertonhttps://github.com/NickWaterton I've made some further API changes today. Can you let me know if you're still seeing this issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/flair-systems/flair-api-client-py/issues/2#issuecomment-476866917, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALCUZxNySmEdYm2p0nfcIMEc2WIClZNbks5vapYzgaJpZM4b9HiS.

Unsubscribe from GE's commercial electronic messages: http://sc.ge.com/*casl-unsubscribe Désabonner des messages électroniques commerciaux de GE: http://sc.ge.com/*lcap-desabonnement

edpaget commented 5 years ago

I made changes to our servers that I was hoping would resolve the issue. Are you seeing your 5 second timeout expiring every 20 minutes or so? Or are you seeing the truncated API responses? I think the current setup you could occasionally see longer than 5s response times, but they should still complete.

NickWaterton commented 5 years ago

I’m seeing the 5 second timeout every 5-20 minutes. This is improved over what it was.

I can send you my logs when I get home, flight gets in tonight.

Nick Waterton P.Eng Sent from my iPhone

On Mar 28, 2019, at 6:39 AM, Edward Paget notifications@github.com<mailto:notifications@github.com> wrote:

I made changes to our servers that I was hoping would resolve the issue. Are you seeing your 5 second timeout expiring every 20 minutes or so? Or are you seeing the truncated API responses? I think the current setup you could occasionally see longer than 5s response times, but they should still complete.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/flair-systems/flair-api-client-py/issues/2#issuecomment-477577737, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALCUZ9lBfGwQLBp6f6Yrtp8RJfCvYeLwks5vbLgLgaJpZM4b9HiS.

Unsubscribe from GE's commercial electronic messages: http://sc.ge.com/*casl-unsubscribe Désabonner des messages électroniques commerciaux de GE: http://sc.ge.com/*lcap-desabonnement

NickWaterton commented 5 years ago

Here is my log, if you search for “Error”, you can see how frequently I have to retry the query due to timeouts.

5.0 secs was just a guess, although why a server would take longer than a few fractions of a second to process the request I don’t know.

Nick Waterton P.Eng. National Support Leader - Nuclear Medicine, PET and RP GE Healthcare

M+1 416 859 8545 E nick.waterton@med.ge.commailto:nick.waterton@med.ge.com

2300 Meadowvale Boulevard Mississauga, ON L5N 5P9 Canada

GE imagination at work

From: Edward Paget notifications@github.com Sent: Wednesday, March 27, 2019 7:55 PM To: flair-systems/flair-api-client-py flair-api-client-py@noreply.github.com Cc: Waterton, Nick (GE Healthcare) Nick.Waterton@med.ge.com; Mention mention@noreply.github.com Subject: EXT: Re: [flair-systems/flair-api-client-py] API Keeps failing to retrieve data (#2)

I made changes to our servers that I was hoping would resolve the issue. Are you seeing your 5 second timeout expiring every 20 minutes or so? Or are you seeing the truncated API responses? I think the current setup you could occasionally see longer than 5s response times, but they should still complete.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/flair-systems/flair-api-client-py/issues/2#issuecomment-477413481, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALCUZ3Kfphe_sTecQfeye-esmw_Kemusks5vbCD_gaJpZM4b9HiS.

Unsubscribe from GE's commercial electronic messages: http://sc.ge.com/*casl-unsubscribe Désabonner des messages électroniques commerciaux de GE: http://sc.ge.com/*lcap-desabonnement