search

cyberjunky / home-assistant-garmin_connect

The Garmin Connect integration allows you to expose data from Garmin Connect to Home Assistant.
MIT License
250 stars 36 forks source link

'Connection pool is full' - regular log flood #85

Open pejotigrek opened 1 year ago

pejotigrek commented 1 year ago

I know that connect.garmin.com is limiting connections, but is there any way to change integration's connection frequency via settings - or maybe is there any chance for adding that [because AFAIK there are no settings available after succesfully starting the integration]?

I am getting lots of connection discards in the log:

2023-06-06 20:22:45.830 WARNING (SyncWorker_10) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:45.833 WARNING (SyncWorker_0) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.091 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.165 WARNING (SyncWorker_23) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.276 WARNING (SyncWorker_13) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.379 WARNING (SyncWorker_19) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.394 WARNING (SyncWorker_20) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.396 WARNING (SyncWorker_16) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.424 WARNING (SyncWorker_17) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.571 WARNING (SyncWorker_24) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.644 WARNING (SyncWorker_18) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.646 WARNING (SyncWorker_22) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.647 WARNING (SyncWorker_21) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
2023-06-06 20:22:46.650 WARNING (SyncWorker_15) [urllib3.connectionpool] Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10

about every 10 minutes or so. only 7 sensors from integration are turned on, so I guess there's no overload with data. is there anything I can do to prevent those warnings? no other integration is causing them.

RaeesD commented 1 year ago

Same issue here, detail below:

Logger: urllib3.connectionpool
Source: /usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py:305
First occurred: June 5, 2023 at 10:07:35 AM (2553 occurrences)
Last logged: 10:08:30 AM
Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10
cyberjunky commented 1 year ago

I cannot reproduce it, nor have I seen these errors. Can you give some details on the home assistant version you run, latest Garmin Connect integration version? And how many Garmin Connect accounts you have in use within HA?

pejotigrek commented 1 year ago

I cannot reproduce it, nor have I seen these errors. Can you give some details on the home assistant version you run, latest Garmin Connect integration version? And how many Garmin Connect accounts you have in use within HA?

my detailed info:

right now I made a "quick-n-dirty" patch in my config, just to filter out those warnings as my logfile was growing too fast - I just added this to my HA configuration:

logger:
  default: warn
  filters:
    urllib3.connectionpool:
    - "Connection pool is full, discarding connection: connect.garmin.com. Connection pool size: 10"

but of course it's just hiding the output, not fixing the problem

cyberjunky commented 1 year ago

There are/were quite a few integrations with same problem https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aclosed+%22Connection+pool+is+full%2C+discarding+connection%22+

Some mention implementing 'DataUpdateCoordinator' is a fix, but we alreay have that... will try to see what other causes were discovered...

fredderoncq77 commented 1 year ago

Hello, I have the same pool problem. I have multiple Garmin devices associated with my Garmin account. Maybe that's a clue? Thanks for your help

fatuuse commented 11 months ago

+1

Version core-2023.12.3
Installationstype   Home Assistant Container
Docker  true
User    root
Python-version  3.11.6
System  Linux
Version 4.4.180+

1 account of Garmin Connect

cyberjunky commented 11 months ago

I just checked my log, no single connection full error with 85 garmin sensors active, runs since last HA update, can one of you guys enable debug and see if it reveals any other errors, or erroneous amount of data gathering from garmin.com? https://github.com/cyberjunky/home-assistant-garmin_connect?tab=readme-ov-file#debugging

RaeesD commented 11 months ago

I got the below after enabling debug log, doesn't seem very helpful (HA core 2023.12.0, supervisor 2023.11.6, Garmin Connect 0.2.17, 1 account):

2023-12-18 16:03:20.498 WARNING (SyncWorker_25) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:03:20.503 WARNING (SyncWorker_19) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:03:20.511 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:08:24.184 WARNING (SyncWorker_1) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:08:24.189 WARNING (SyncWorker_18) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:08:24.189 WARNING (SyncWorker_17) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:13:28.167 WARNING (SyncWorker_13) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:13:28.181 WARNING (SyncWorker_22) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:13:28.187 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:18:29.869 DEBUG (MainThread) [custom_components.garmin_connect] {'userProfileId': 31779493, 'totalKilocalories': 764.0, 'activeKilocalories': 29.0, 'bmrKilocalories': 735.0, 'wellnessKilocalories': 764.0, 'burnedKilocalories': None, 'consumedKilocalories': None, 'remainingKilocalories': 764.0, 'totalSteps': 654, 'netCalorieGoal': 2320, 'totalDistanceMeters': 548, 'wellnessDistanceMeters': 548, 'wellnessActiveKilocalories': 29.0, 'netRemainingKilocalories': 2349.0, 'userDailySummaryId': 31779493, 'calendarDate': '2023-12-18', 'rule': {'typeId': 1, 'typeKey': 'public'}, 'uuid': '(redacted)', 'dailyStepGoal': 1000, 'wellnessStartTimeGmt': '2023-12-17T22:00:00.0', 'wellnessStartTimeLocal': '2023-12-18T00:00:00.0', 'wellnessEndTimeGmt': '2023-12-18T06:53:00.0', 'wellnessEndTimeLocal': '2023-12-18T08:53:00.0', 'durationInMilliseconds': 31980000, 'wellnessDescription': None, 'highlyActiveSeconds': 300, 'activeSeconds': 1079, 'sedentarySeconds': 8941, 'sleepingSeconds': 21660, 'includesWellnessData': True, 'includesActivityData': False, 'includesCalorieConsumedData': False, 'privacyProtected': False, 'moderateIntensityMinutes': 0, 'vigorousIntensityMinutes': 0, 'floorsAscendedInMeters': 10.683, 'floorsDescendedInMeters': 0.0, 'floorsAscended': 3.50492, 'floorsDescended': 0.0, 'intensityMinutesGoal': 180, 'userFloorsAscendedGoal': 10, 'minHeartRate': 51, 'maxHeartRate': 113, 'restingHeartRate': 53, 'lastSevenDaysAvgRestingHeartRate': 53, 'source': 'GARMIN', 'averageStressLevel': 18, 'maxStressLevel': 91, 'stressDuration': 3120, 'restStressDuration': 24360, 'activityStressDuration': 1620, 'uncategorizedStressDuration': 2880, 'totalStressDuration': 31980, 'lowStressDuration': 2460, 'mediumStressDuration': 420, 'highStressDuration': 240, 'stressPercentage': 9.76, 'restStressPercentage': 76.17, 'activityStressPercentage': 5.07, 'uncategorizedStressPercentage': 9.01, 'lowStressPercentage': 7.69, 'mediumStressPercentage': 1.31, 'highStressPercentage': 0.75, 'stressQualifier': 'UNKNOWN', 'measurableAwakeDuration': 7440, 'measurableAsleepDuration': 21660, 'lastSyncTimestampGMT': '2023-12-18T06:53:53.955', 'minAvgHeartRate': 52, 'maxAvgHeartRate': 112, 'bodyBatteryChargedValue': 53, 'bodyBatteryDrainedValue': 16, 'bodyBatteryHighestValue': 85, 'bodyBatteryLowestValue': 32, 'bodyBatteryMostRecentValue': 69, 'bodyBatteryDuringSleep': None, 'bodyBatteryVersion': 2.0, 'abnormalHeartRateAlertsCount': None, 'averageSpo2': 93.0, 'lowestSpo2': 83, 'latestSpo2': 95, 'latestSpo2ReadingTimeGmt': '2023-12-18T04:00:00.0', 'latestSpo2ReadingTimeLocal': '2023-12-18T06:00:00.0', 'averageMonitoringEnvironmentAltitude': 16.0, 'restingCaloriesFromActivity': None, 'avgWakingRespirationValue': 13.0, 'highestRespirationValue': 16.0, 'lowestRespirationValue': 8.0, 'latestRespirationValue': 14.0, 'latestRespirationTimeGMT': '2023-12-18T06:53:00.0'}
2023-12-18 16:18:30.786 DEBUG (MainThread) [custom_components.garmin_connect] {'startDate': '2023-12-18', 'endDate': '2023-12-18', 'dateWeightList': [], 'totalAverage': {'from': 1702857600000, 'until': 1702943999999, 'weight': None, 'bmi': None, 'bodyFat': None, 'bodyWater': None, 'boneMass': None, 'muscleMass': None, 'physiqueRating': None, 'visceralFat': None, 'metabolicAge': None}}
2023-12-18 16:18:31.680 DEBUG (MainThread) [custom_components.garmin_connect] [{'alarmMode': 'ON', 'alarmTime': 360, 'alarmDays': ['M', 'Tu', 'W', 'Th', 'F'], 'alarmSound': 'TONE_AND_VIBRATION', 'alarmId': 1612192333, 'changeState': 'UNCHANGED', 'backlight': None, 'enabled': None, 'alarmMessage': None, 'alarmImageId': None, 'alarmIcon': None, 'alarmType': None}, {'alarmMode': 'OFF', 'alarmTime': 300, 'alarmDays': ['ONCE'], 'alarmSound': 'TONE_AND_VIBRATION', 'alarmId': 1644610597, 'changeState': 'UNCHANGED', 'backlight': None, 'enabled': None, 'alarmMessage': None, 'alarmImageId': None, 'alarmIcon': None, 'alarmType': None}, {'alarmMode': 'ON', 'alarmTime': 300, 'alarmDays': ['Sa', 'Su'], 'alarmSound': 'VIBRATION', 'alarmId': 1648924787, 'changeState': 'UNCHANGED', 'backlight': None, 'enabled': None, 'alarmMessage': None, 'alarmImageId': None, 'alarmIcon': None, 'alarmType': None}]
2023-12-18 16:18:32.496 WARNING (SyncWorker_7) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:18:32.504 WARNING (SyncWorker_14) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:18:32.516 WARNING (SyncWorker_19) [urllib3.connectionpool] Connection pool is full, discarding connection: connectapi.garmin.com. Connection pool size: 10
2023-12-18 16:18:33.587 DEBUG (MainThread) [custom_components.garmin_connect] Finished fetching garmin_connect data in 4.038 seconds (success: True)
cyberjunky commented 11 months ago

Thanks, I suspect there are maybe other integrations using urllib connection pool, or connections aren't cleared/reused for some reason? It tries to poll every 5 minutes which is normal.

cyberjunky commented 11 months ago

I'm searching other closed issues which are similar, it seems there is a setting variable like ' PARALLEL_UPDATES = 2' to limit number of parallel request... or it has to be done in request lib, working on it.. For reference: https://github.com/tuya/tuya-iot-python-sdk/pull/63/files https://github.com/kbr/fritzconnection/pull/114/files

cyberjunky commented 10 months ago

@pejotigrek @fatuuse @RaeesD @fredderoncq77 The package underneath garminconnect called 'garth' has now limited the connections to 10, this solution was found in other home assistant integrations with same issue. Can you test version 0.2.18 to see if this fixed the issue?

fatuuse commented 10 months ago

@pejotigrek @fatuuse @RaeesD @fredderoncq77 The package underneath garminconnect called 'garth' has now limited the connections to 10, ...

Thank you! But no change for me.

pejotigrek commented 10 months ago

@pejotigrek @fatuuse @RaeesD @fredderoncq77 The package underneath garminconnect called 'garth' has now limited the connections to 10, this solution was found in other home assistant integrations with same issue. Can you test version 0.2.18 to see if this fixed the issue?

thanks, unfortunately - no change for me too :(

areeeee commented 1 week ago

Any solution for this?