Warning log connection pool is full, discarding connection

[dMopp]

This error originated from a custom integration.

Logger: urllib3.connectionpool Source: custom_components/proxmoxve/init.py:236 Integration: Proxmox VE (documentation, issues) First occurred: 11:24:22 (1 occurrences) Last logged: 11:24:22

Connection pool is full, discarding connection: . Connection pool size: 10

[dougiteixeira]

This warning log is caused outside of the integration, I'll check if there's a way to deal with it...

[nima-1102]

+1

Integration Version: 3.1.0

Core: 2023.11.2 Supervisor: 2023.11.3 Operating System: 11.1 Frontend: 20231030.2

This error originated from a custom integration.

Logger: urllib3.connectionpool
Source: custom_components/proxmoxve/coordinator.py:404
Integration: Proxmox VE (documentation, issues)
First occurred: 20:13:33 (1 occurrences)
Last logged: 20:13:33

Connection pool is full, discarding connection: 192.168.1.200. Connection pool size: 10

This error originated from a custom integration.

Logger: urllib3.connectionpool
Source: custom_components/proxmoxve/coordinator.py:496
Integration: Proxmox VE (documentation, issues)
First occurred: 20:13:33 (1 occurrences)
Last logged: 20:13:33

Connection pool is full, discarding connection: 192.168.1.200. Connection pool size: 10

[dmshimself]

same for me with HA 2023.12.2

[alexruffell]

What does this error mean, and is there anything we can do on our end to eliminate or reduce its frequency? I have 3 nodes so that may be part of why I see this warning a lot in my logs

[ChristophCaina]

is there any update on this? I just installed the integration today - and my log is pretty full of those warnings... (at least, three in a row)...

In addition to that, I have an error logged about some time outs... ?

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.proxmoxve
Source: helpers/update_coordinator.py:306
Integration: Proxmox VE (documentation, issues)
First occurred: 11:08:54 (7 occurrences)
Last logged: 11:08:54

Timeout fetching proxmox_coordinator_disk_homeserver_/dev/sdc data
Timeout fetching proxmox_coordinator_lxc_122 data
Timeout fetching proxmox_coordinator_disk_homeserver_/dev/sdg data
Timeout fetching proxmox_coordinator_disk_homeserver_/dev/sda data
Timeout fetching proxmox_coordinator_lxc_120 data

[dougiteixeira]

I believe you can filter these logs using the code below in your configuration.yaml file:

logger:
  default: critical
  filters:
    urllib3.connectionpool:
      - "Connection pool is full"

[DDomnick]

So am I to understand the solution to this issue of dropping connections is to hide the error? Isn't there a way to increase the connection pool size >10?

Other report on closed issue that went stale: https://github.com/home-assistant/core/issues/68487

Logger: urllib3.connectionpool
Source: custom_components/proxmoxve/api.py:82
integration: Proxmox VE ([documentation](https://github.com/dougiteixeira/proxmoxve), [issues](https://github.com/dougiteixeira/proxmoxve/issues))
First occurred: 9:27:39 AM (4 occurrences)
Last logged: 9:48:39 AM
Connection pool is full, discarding connection: 10.10.0.121. Connection pool size: 10

[dMopp]

Hiding is not a solution… Is this a ha core issue or an addon issue? Proxmox addon has a very slow starting time and tons of requests. Maybe optimizing the connections (open, use, close) can solve that, too

[dougiteixeira]

The suggestion to hide the logs is just a stopgap solution, the definitive solution depends on a deeper analysis (this same problem occurs with some main Home Assistant integrations), at the moment I am not available to delve deeper into this. Feel free to investigate and propose a definitive solution to the problem.

[DDomnick]

Thank you for the update, additional information, and certainly your work on this excellent integration. I didn't realize this issue was systemic. I'll take a deeper dive, and will update if i find a more permanent solution.

---

From: dougiteixeira @.***> Sent: Tuesday, April 9, 2024 10:01 PM To: dougiteixeira/proxmoxve Cc: NTI Distribution List; Comment Subject: Re: [dougiteixeira/proxmoxve] Warning log connection pool is full, discarding connection (Issue #76)

The suggestion to hide the logs is just a stopgap solution, the definitive solution depends on a deeper analysis (this same problem occurs with some main Home Assistant integrations), at the moment I am not available to delve deeper into this. Feel free to investigate and propose a definitive solution to the problem.

— Reply to this email directly, view it on GitHubhttps://github.com/dougiteixeira/proxmoxve/issues/76#issuecomment-2046321937, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOHZAO7MFLMUZQLTM3QDU3LY4SMNZAVCNFSM6AAAAAAXQZX6P2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBWGMZDCOJTG4. You are receiving this because you commented.Message ID: @.***>

[ejpenney]

If I understand correctly, the underlying problem is the connection pool isn't large enough. You can see here a brute-force solution to this problem. I just did this on my development server, usually I get connection pool messages from Proxmox approximately every 90 seconds after boot, the dev server is getting zero messages for 30 minutes and running.

Two things become immediately apparent:

1. I have no idea what the correct number of connections to have in the pool is, is this dependent on our setup? I've been looking at this for all-of ten minutes so maybe the answer is obvious to someone else. There may be a static number, or it may need to be something dependent on a specific Home Assistant setup (EG: number of guests we're monitoring?)
2. This appears to be a problem in proxmoxer, the upstream library, I had to patch a private object in this library to get this hack working. A "proper" fix, would be to either have proxmoxer set the pool size appropriately, or allow this integration to pass a desired pool size into the constructor.

Anyways, I hope this helps someone who knows a bit more about these specific projects...

[jsapede]

@ejpenney POC did the trick ! will have some long time test to see but wherever i look, proxmoxer or urllib3 they only say to hide the warnings :(