Closed timmmmmey closed 5 years ago
Hi timmmmey, it looks like a memory-leak. Reason could be the many connection - requests of socket-clients. Please look for that portnumber: 8088 in your system. It could be that it is already used for something else. So change that port-numer (for server and client) in the configuration-File: ~/HT3/sw/etc/config/ht_proxy_cfg.xml to any other unused port-number and start the Pi again.
Hi Norbert,
the port is only used by ht_proxy and and the only connection initiated by the fhem heatronic plugin. Changing the port didn't help (as long I change it in fhem as well).
Every ~10 minutes there is also a short time connection from ht_netclient (cronjob, because fhem plugin sending temperatures is not working well and commands from the ht_netclient sometimes got ignored from the heater, so I put this in a cron job)
Your memory-leak theorie seems to fit, my ht_proxy is occupying 1,6GB virtual memory right now. I know virtual memory is not the real used memory, so I will try to run it with valgrid/massif and post the results afterwards.
--------------------------------------------------------------------------------
Command: python3 /opt/ht3/ht_proxy.py
Massif arguments: --massif-out-file=/opt/ht3/mem_prof
ms_print arguments: /opt/ht3/mem_prof
--------------------------------------------------------------------------------
MB
1.158^ #
| #
| #:::::::::::::@@@::::::::::::::::::@::::@:::::@::::
| : @#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| ::::::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| ::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| : @:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| ::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| ::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :::::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| @::::::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| @::::::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| @::::::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
| :@::::::@:::::: :::@#::: ::: : : @ : :: ::: :::::::::@: ::@:::::@::::
0 +----------------------------------------------------------------------->Mi
0 368.9
Number of snapshots: 67
Detailed snapshots: [2, 9, 19, 20 (peak), 29, 46, 50, 60]
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 0 0 0 0
1 7,910,915 118,320 115,806 2,514 0
2 12,608,269 283,384 279,722 3,662 0
98.71% (279,722B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->53.11% (150,496B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->13.65% (38,686B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->12.52% (35,472B) 0xDBCFA: _PyBytes_Resize (in /usr/bin/python3.4)
|
->05.74% (16,270B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->05.10% (14,448B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->04.26% (12,062B) in 47 places, all below massif's threshold (01.00%)
|
->02.89% (8,192B) 0xC76E2: _PyFaulthandler_Init (in /usr/bin/python3.4)
|
->01.45% (4,096B) 0xD350A: ??? (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
3 17,035,122 262,976 258,584 4,392 0
4 24,699,125 339,792 333,949 5,843 0
5 29,400,944 361,952 355,150 6,802 0
6 34,929,209 469,112 461,906 7,206 0
7 41,417,527 616,424 607,192 9,232 0
8 46,100,465 604,240 594,325 9,915 0
9 51,661,824 642,496 631,829 10,667 0
98.34% (631,829B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->54.07% (347,392B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.42% (73,392B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->11.37% (73,078B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->03.52% (22,624B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.97% (19,088B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.63% (16,896B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->02.55% (16,396B) in 64 places, all below massif's threshold (01.00%)
|
->02.39% (15,387B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->01.91% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.48% (9,512B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.46% (9,376B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.28% (8,208B) 0xE30D2: PyArena_New (in /usr/bin/python3.4)
|
->01.28% (8,192B) 0xC76E2: _PyFaulthandler_Init (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
10 55,333,195 657,464 646,550 10,914 0
11 60,258,708 665,904 654,460 11,444 0
12 66,568,795 714,608 702,036 12,572 0
13 71,670,640 760,240 746,923 13,317 0
14 78,582,509 928,584 914,396 14,188 0
15 84,587,520 964,208 949,291 14,917 0
16 92,100,037 1,049,624 1,033,666 15,958 0
17 97,332,672 992,296 975,640 16,656 0
18 104,197,582 1,027,240 1,009,175 18,065 0
19 110,490,727 1,040,216 1,021,899 18,317 0
98.24% (1,021,899B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->57.22% (595,168B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.44% (129,446B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->08.29% (86,198B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->04.14% (43,024B) in 92 places, all below massif's threshold (01.00%)
|
->03.62% (37,694B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.18% (33,040B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.14% (22,304B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.09% (21,760B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.45% (15,120B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.33% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.18% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.16% (12,032B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
20 115,912,173 1,214,728 1,194,864 19,864 0
98.36% (1,194,864B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->49.88% (605,856B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.54% (152,267B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->10.66% (129,446B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->05.57% (67,601B) 0xAEF42: ??? (in /usr/bin/python3.4)
|
->05.28% (64,116B) in 96 places, all below massif's threshold (01.00%)
|
->02.97% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->02.85% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.79% (33,936B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.36% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.33% (16,208B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.14% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.01% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
21 121,511,018 1,094,256 1,074,388 19,868 0
22 127,653,812 1,094,256 1,074,388 19,868 0
23 133,792,713 1,094,256 1,074,388 19,868 0
24 139,928,584 1,094,256 1,074,388 19,868 0
25 146,075,437 1,094,256 1,074,388 19,868 0
26 150,984,717 1,094,256 1,074,388 19,868 0
27 162,616,889 1,098,368 1,078,212 20,156 0
28 176,263,811 1,098,344 1,078,197 20,147 0
29 192,230,493 1,094,576 1,074,444 20,132 0
98.16% (1,074,444B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.66% (609,232B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->12.04% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->08.03% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.18% (56,748B) in 101 places, all below massif's threshold (01.00%)
|
->03.29% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.18% (34,832B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.16% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.62% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.52% (16,688B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.26% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.12% (12,288B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.08% (11,840B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
30 209,428,136 1,094,568 1,074,444 20,124 0
31 216,061,069 1,107,736 1,087,072 20,664 0
32 221,020,184 1,107,392 1,086,724 20,668 0
33 225,220,705 1,107,408 1,086,736 20,672 0
34 233,501,124 1,107,328 1,086,660 20,668 0
35 239,678,908 1,107,312 1,086,656 20,656 0
36 245,818,744 1,107,312 1,086,656 20,656 0
37 254,080,310 1,107,376 1,086,720 20,656 0
38 262,279,318 1,107,424 1,086,740 20,684 0
39 268,565,366 1,107,280 1,086,640 20,640 0
40 272,750,390 1,107,280 1,086,640 20,640 0
41 276,853,362 1,107,352 1,086,676 20,676 0
42 283,014,570 1,107,328 1,086,660 20,668 0
43 287,158,030 1,107,296 1,086,644 20,652 0
44 295,381,273 1,107,576 1,086,904 20,672 0
45 299,485,221 1,107,560 1,086,900 20,660 0
46 305,649,852 1,107,544 1,086,896 20,648 0
98.14% (1,086,896B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.06% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.90% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.93% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.79% (64,096B) in 102 places, all below massif's threshold (01.00%)
|
->03.25% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.59% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.57% (17,440B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.48% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.25% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.06% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
47 309,826,025 1,107,584 1,086,916 20,668 0
48 318,071,099 1,107,608 1,086,932 20,676 0
49 324,240,506 1,107,600 1,086,920 20,680 0
50 327,920,023 1,107,544 1,086,896 20,648 0
98.14% (1,086,896B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->55.06% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.90% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.93% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->05.79% (64,096B) in 102 places, all below massif's threshold (01.00%)
|
->03.25% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->03.12% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->02.59% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.57% (17,440B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.48% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.25% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.06% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
51 331,614,300 1,120,536 1,099,893 20,643 0
52 335,296,269 1,115,928 1,095,480 20,448 0
53 338,984,448 1,115,928 1,095,480 20,448 0
54 342,659,845 1,115,888 1,095,460 20,428 0
55 346,348,334 1,115,888 1,095,460 20,428 0
56 350,046,237 1,115,928 1,095,468 20,460 0
57 353,759,198 1,115,928 1,095,480 20,448 0
58 357,437,992 1,115,912 1,095,476 20,436 0
59 361,126,476 1,115,928 1,095,480 20,448 0
60 364,801,506 1,115,912 1,095,476 20,436 0
98.17% (1,095,476B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->54.65% (609,840B) 0x690CA: ??? (in /usr/bin/python3.4)
|
->11.81% (131,828B) 0x6014A: PyUnicode_New (in /usr/bin/python3.4)
|
->07.87% (87,842B) 0x5FC96: PyObject_Malloc (in /usr/bin/python3.4)
|
->06.59% (73,524B) in 102 places, all below massif's threshold (01.00%)
|
->03.23% (36,041B) 0x83142: PyBytes_FromStringAndSize (in /usr/bin/python3.4)
|
->03.10% (34,608B) 0x9B826: _PyDict_NewPresized (in /usr/bin/python3.4)
|
->03.09% (34,496B) 0x5E856: PyDict_New (in /usr/bin/python3.4)
|
->02.57% (28,672B) 0xD350A: ??? (in /usr/bin/python3.4)
|
->01.50% (16,704B) 0xB328A: ??? (in /usr/bin/python3.4)
|
->01.47% (16,384B) 0x103ECE: ??? (in /usr/bin/python3.4)
|
->01.24% (13,825B) 0xAE812: ??? (in /usr/bin/python3.4)
|
->01.05% (11,712B) 0x9AFAE: _PyDict_NewKeysForClass (in /usr/bin/python3.4)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
61 368,491,931 1,115,928 1,095,480 20,448 0
62 372,163,591 1,115,904 1,095,464 20,440 0
63 375,849,820 1,115,928 1,095,468 20,460 0
64 379,525,175 1,115,904 1,095,464 20,440 0
65 383,196,627 1,115,936 1,095,480 20,456 0
66 386,871,544 1,116,632 1,096,168 20,464 0
Scheint ein memory-leak zu sein. Jedoch nutzt definitiv nichts anderes als das FHEM-Plugin und der ht_netclient (wobei der häufig gestartet wird ~ alle 10 Minuten). Irgendwie scheint die Verbindung nicht richtig zu terminieren und daher die Ressourcen nicht freigegeben oder so. Kann ich noch irgendwas prüfen?
Hi timmmmmey, it seems to be a bit strange for me. This lack of memory behaviour running the 'ht_proxy.server' I haven't seen on my maschine. On my RPI B it runs years so far. On other maschines too. To fix that problem on your maschine please do following: 1.Deactivate the HT3-applikation under your folder /opt. It runs there as user 'root' and that environment could be a bit different.
Let me ask: Which type of RPI you are using? Which type of ht_adapter you are using?
Hi,
recently some exceptions occur in ht_rpxy
Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 43732) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 43734) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 54694) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 54696) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 55762) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 37414) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ---------------------------------------- Apr 04 19:58:07 fhempi ht_proxy.py[2112]: Exception happened during processing of request from ('127.0.0.1', 37416) Apr 04 19:58:07 fhempi ht_proxy.py[2112]: ----------------------------------------
which leads to the following error:
Apr 04 20:18:07 fhempi ht_proxy.py[2112]: ---------------------------------------- File "lib/ht_proxy_if.py", line 409, in add_client Apr 04 20:18:07 fhempi ht_proxy.py[2112]: txThread.start() Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "/usr/lib/python3.4/threading.py", line 850, in start Apr 04 20:18:07 fhempi ht_proxy.py[2112]: _start_new_thread(self._bootstrap, ()) Apr 04 20:18:07 fhempi ht_proxy.py[2112]: RuntimeError: can't start new thread Apr 04 20:18:07 fhempi ht_proxy.py[2112]: Traceback (most recent call last): Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "/usr/lib/python3.4/socketserver.py", line 613, in process_request_thread Apr 04 20:18:07 fhempi ht_proxy.py[2112]: self.finish_request(request, client_address) Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request Apr 04 20:18:07 fhempi ht_proxy.py[2112]: self.RequestHandlerClass(request, client_address, self) Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "/usr/lib/python3.4/socketserver.py", line 669, in __init__ Apr 04 20:18:07 fhempi ht_proxy.py[2112]: self.handle() Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "lib/ht_proxy_if.py", line 445, in handle Apr 04 20:18:07 fhempi ht_proxy.py[2112]: _ClientHandler.add_client(self._myownID, self.request) Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "lib/ht_proxy_if.py", line 409, in add_client Apr 04 20:18:07 fhempi ht_proxy.py[2112]: txThread.start() Apr 04 20:18:07 fhempi ht_proxy.py[2112]: File "/usr/lib/python3.4/threading.py", line 850, in start Apr 04 20:18:07 fhempi ht_proxy.py[2112]: _start_new_thread(self._bootstrap, ())
maybe we should close the thread on an exception or use a threadpool? Is there any possibility to get more infomation about the exception?