jasonacox / Powerwall-Dashboard

Grafana Monitoring Dashboard for Tesla Solar and Powerwall Systems
MIT License
296 stars 63 forks source link

Timeout In Docker InfluxDB Write #340

Closed aesculus closed 1 year ago

aesculus commented 1 year ago

@jasonacox Sorry for a unrelated issue but thought you might have some insight. I am using the weather server example to create another dataset into the powerwall db with a Docker container. All works well with the script running in a shell.

But when I build a Docker container I am having a issue during the write where it times out.

INFO:Error writing to InfluxDB
INFO:(<urllib3.connection.HTTPConnection object at 0x7ff9e3365990>, 'Connection to 192.168.1.108 timed out. (connect timeout=10.0)')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
TimeoutError: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/server.py", line 208, in api_poller_task
    write_api.write(IDB,IORG,output)
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/client/write_api.py", line 374, in write
    results = list(map(write_payload, payloads.items()))
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/client/write_api.py", line 372, in write_payload
    return self._post_write(_async_req, bucket, org, final_string, payload[0])
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/client/write_api.py", line 483, in _post_write
    return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/service/write_service.py", line 62, in post_write
    (data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/service/write_service.py", line 92, in post_write_with_http_info
    return self.api_client.call_api(
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
    response_data = self.request(
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
    return self.rest_client.POST(url,
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/_sync/rest.py", line 307, in POST
    return self.request("POST", url,
  File "/usr/local/lib/python3.10/site-packages/influxdb_client/_sync/rest.py", line 216, in request
    r = self.pool_manager.request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/_request_methods.py", line 118, in request
    return self.request_encode_body(
  File "/usr/local/lib/python3.10/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python3.10/site-packages/urllib3/poolmanager.py", line 443, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 445, in increment
    raise reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 395, in request
    self.endheaders()
  File "/usr/local/lib/python3.10/http/client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.10/http/client.py", line 1038, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.10/http/client.py", line 976, in send
    self.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 243, in connect
    self.sock = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 212, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPConnection object at 0x7ff9e3365990>, 'Connection to 192.168.1.108 timed out. (connect timeout=10.0)')

I have tried different versions of the pip install influxdb-client thinking it could have been something new that was introduced recently but no changes.

Any thoughts or ideas to try?

jasonacox commented 1 year ago

Hi Chris, there appears to be something off in how you are creating or running the container. Docker namespaces the network too so it will need to plumb the access between the container and influxdb.

Can you share your Dockerfile you use to create the container, and details on how you are running it (e.g. script or docker compose)?

Are you added the new container to the powerwall.yml?

https://github.com/jasonacox/Powerwall-Dashboard/blob/37a66d10231e3d5c685c5957bf86ea6ac8164834/powerwall.yml#L80-L98

aesculus commented 1 year ago

Hi @jasonacox Thanks for the speedy reply. While I might/probably will add it to the powerwall.yml later, right not I am just trying to get it to run stand alone. Maybe that is a bad assumption the way its constructed.

The container app works great when it's not a container. Connects to influxdb and writes to the db just fine. But when I create it as a container and run it, everything works OK until I try to write the data to the powerwall db. Then it times out.

So I created a simple test case that just tries to ping the Influx server. It too works great when run under the shell but times out the same when built and run as a container in Docker.

Here is my test script (note the app uses asynicio instead of threads but I tested without that too and failed):

#!/usr/bin/env python
# LennoxS30 - Current Lennox S30 Conditions
# -*- coding: utf-8 -*-
import asyncio

from influxdb_client.client.influxdb_client_async import InfluxDBClientAsync

async def main():
    async with InfluxDBClientAsync(url="http://192.168.1.108:8086", token="my-token", org="my-org") as client:
        ready = await client.ping()
        print(f"InfluxDB: {ready}")

if __name__ == "__main__":
    asyncio.run(main())

And I build it with:

docker build -t lennoxs30:latest .

And then make a container and run it with:

docker run \
-d \
-e LENNOXS30CONF='lennoxs30.conf' \
--name lennoxs30 \
--restart unless-stopped \
lennoxs30

And the Docker file:

FROM python:3.10-alpine
WORKDIR /app
RUN pip3 install requests influxdb-client aiohttp aiocsv lennoxs30api pytz
COPY lennoxs30.conf lennoxs30.conf
COPY server.py server.py
CMD ["python3", "server.py"]
jasonacox commented 1 year ago

I see the issue. Your docker run is not putting the container in host mode so it will not be able to hit your private IP address endpoint (http://192.168.1.108:8086)

You should be able to so something like this:

docker run \
-d \
-e LENNOXS30CONF='lennoxs30.conf' \
--name lennoxs30 \
--restart unless-stopped \
--net=host \
lennoxs30

Before you mention it, I see that the weather411 example omits this too, so TO-DO here to update that. 😁

aesculus commented 1 year ago

Nailed it.

I cannot tell you how many hours I spent on the net about this. And this is not a totally unique problem. Could not find any real fixes and not one mentioned this at all. That was surprising given how obvious it was after the fact.

Chalk this up to my total unfamiliarity with Docker and just following your lead with the Weather sample and hacking away at it via trial and error. So in the end we fixed a small oversight so the day can be considered a success.