Unable to start due to thread error

[canepan]

Hi, I just cloned the repo, customized docker-compose.yaml and tried a docker-compose up, but this fails with this error:

pyfusionsolardatarelay | Traceback (most recent call last):
pyfusionsolardatarelay |   File "/code/pv.py", line 33, in <module>
pyfusionsolardatarelay |     fs_thread.start()
pyfusionsolardatarelay |   File "/usr/local/lib/python3.10/threading.py", line 935, in start
pyfusionsolardatarelay |     _start_new_thread(self._bootstrap, ())
pyfusionsolardatarelay | RuntimeError: can't start new thread

From the Python threads I found this could be related to too many threads running, but I think there is only one in my case...

I also tried on a Raspberry Pi (which would be my preferred way to keep the container running), but it didn't start with this error:

pyfusionsolardatarelay | standard_init_linux.go:219: exec user process caused: exec format error

Is there a way to have either one to start? I removed all containers except the pvfusionsolar one, since I only want to publish to pvoutput.org I changed the folloving variables:

• pvsysname
• pvfusionsolarkkid
• pvfusionsolarurl (added, since it was changed recently to https://uni004eu5.fusionsolar.huawei.com/pvmswebsite/nologin/assets/build/cloud.html#/kiosk?kk= for me)
• pvinflux=False
• pvpvoutputsystemid
• pvpvoutputapikey
• pvmqtt=False

Thanks in any case!

[JasperE84]

Thats the wrong URL, it should point to the API, not the HTML page of the kiosk.

1. Open your kiosk page in chrome,
2. Right click the page an click inspect.
3. Open the "Network" tab
4. Refresh the kiosk
5. Set a search filter for "station-kiosk" in the search box in the network tab
6. Copy the URL in the found network resource
7. That's the url you need

If debug is enabled it should try to fetch and parse that API in the initialization of the thread. That was recently added. Maybe I'm missing some error logging / debug messages so we don't get to see the actual error in a descriptive log message. I'm guessing that an exception on the parse causes the exception not to be logged and i should wrap lines 31-33 in pv.py in a descriptive try/catch with error log on a next update

[JasperE84]

" RuntimeError: can't start new thread" does seem to be a system error, rather than a problem with this code. More people using docker and python threads seem to experience this issue. Maybe the solution is among the replies to the thread here on SO? https://stackoverflow.com/questions/70087344/python-in-docker-runtimeerror-cant-start-new-thread

[canepan]

Hi, I was about to post the same. I did the workaround with privileged: true, but it seems the correct way would be to change the base image in Dockerfile to have -bullseye at the end.

Any solution to run this on arm/arm64?

[canepan]

How can this be closed without a fix?

[JasperE84]

First of all its not closed. Second, it seems to be a problem with your system/os/docker version rather than this repository. If the bug is in another piece of software the fix should be made there not here.

Anyway when I have some time I'll go and see if my system is also affected if i rebuild the docker images with latest python base

[canepan]

Sorry: I saw the "mentioned" appear while writing...

The problem is Docker compatibility with the base image (according to this thread). Would it be possible to rebuild the image with python:3.10-slim-bullseye?

[canepan]

First of all its not closed. Second, it seems to be a problem with your system/os/docker version rather than this repository. If the bug is in another piece of software the fix should be made there not here.

Anyway when I have some time I'll go and see if my system is also affected if i rebuild the docker images with latest python base

I think unless you rebuild your local image, you won't be affected

[canepan]

One more note: after rebuilding the image on my raspberry, it now works perfectly (without privileged: true), since I have Docker 20.10.5 there. Do you think it would be possible to upload the arm image with the same name to help others like me?

Thanks for the amazing work and the quick responses!

[JasperE84]

So I've checked if this is an issue I can reproduce, but I didn't get any issues. I think it might be something local to your OS/docker version.

Pinning the docker repo version to bullseye doesn't seem like the best solution to me since bullseye will be end of support in '26. Probably better to set the privileged workaround on affected systems until the underlying issue is solved.

Let's keep this issue open to see if others are affected. (I'd like to hear from them if they are) Thanks for all the feedback!

The cmd's below seem to completely rebuild (i watched the apt processes scroll by and it took some 23secs on my system)

$ docker-compose stop
$ docker-compose rm -f
$ docker-compose pull
$ docker-compose build --no-cache

Building pyfusionsolar
[+] Building 23.9s (13/13) FINISHED                                                                                                            docker:default
 => [internal] load build definition from Dockerfile                                                                                                     0.0s
 => => transferring dockerfile: 399B                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim                                                                                      0.0s
 => [1/8] FROM docker.io/library/python:3.10-slim                                                                                                        0.0s
 => [internal] load build context                                                                                                                        0.1s
 => => transferring context: 29.27kB                                                                                                                     0.1s
 => CACHED [2/8] WORKDIR /code                                                                                                                           0.0s
 => [3/8] RUN apt-get clean && apt-get update && apt-get upgrade -y                                                                                     12.4s
 => [4/8] RUN pip install --upgrade pip                                                                                                                  4.2s
 => [5/8] RUN pip install --upgrade setuptools                                                                                                           2.3s
 => [6/8] COPY requirements.txt requirements.txt                                                                                                         0.1s
 => [7/8] RUN pip install -r requirements.txt                                                                                                            3.8s
 => [8/8] COPY . .                                                                                                                                       0.3s
 => exporting to image                                                                                                                                   0.7s
 => => exporting layers                                                                                                                                  0.7s
 => => writing image sha256:87fb8ec63cf03c08cd2979cd4a02a3c5fb7af743efa9cf49ca45df45aac841f4                                                             0.0s

 => => naming to docker.io/library/docker_vmetrics_pyfusionsolar               

$ docker-compose up -d
[cut]
$ docker logs -f pyfusionsolardatarelay
[cut] 
2024-02-10 21:10:44,857 - root - DEBUG - PvRelay class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvFusionSolar class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvOutputOrg class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvMqtt class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvInflux class instantiated
2024-02-10 21:10:44,857 - root - INFO - Starting PvRelay on separate thread
2024-02-10 21:10:44,857 - root - DEBUG - PvRelay waiting 5sec to initialize docker-compose containers
2024-02-10 21:10:44,857 - root - DEBUG - GridRelay class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - GridKenter class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvOutputOrg class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvMqtt class instantiated
2024-02-10 21:10:44,857 - root - DEBUG - PvInflux class instantiated
2024-02-10 21:10:44,857 - root - INFO - Starting GridRelay on separate thread
2024-02-10 21:10:44,857 - root - DEBUG - GridRelay waiting 5sec to initialize docker-compose containers
2024-02-10 21:10:49,859 - root - INFO - Starting process_fusionsolar_request() at init, before waiting for cron, because we're in debug mode
2024-02-10 21:10:49,860 - root - INFO - Requesting data from FusionSolar Kiosk API...

[JasperE84]

Also, It'd probably be a good idea to find out how to publish ARM packages or docker builds tagged for bullseye specific version if more people are affected. For now though they need to locally build the dockerfile as I'm lacking the time to set allt hat up. Glad we've found a workaround for the meantime!