Open GoogleCodeExporter opened 9 years ago
let me add that the sqlite database contained only today's data when I did that
experiment
Original comment by mennu...@debian.org
on 29 Mar 2013 at 6:59
Hi,
This is my wfrog box (ALIX 3D2 - 256Mb RAM) with 3 years of data (5 minutes
samples) using a Firebird database and reporting to Wunderground,
meteoclimatic, pwsweather and generating a web page via FTP (10 minutes update):
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2873 root 20 0 42708 12m 4496 S 1.0 5.1 98:40.31 wfrog
20745 root 20 0 56384 18m 4564 S 0.3 7.2 48:39.81 wfrog
2843 firebird 20 0 79584 5512 3396 S 0.0 2.1 20:13.56 fbserver
I am running render and logger in separate processes. Memory consumption looks
coherent with your numbers. Most wfrog classes are instantiated through yaml
configuration files, so it is likely that just doing imports of wfrog modules
is not equivalent.
Jordi.
Original comment by jordi.pu...@gmail.com
on 29 Mar 2013 at 7:12
hi,
I looked a bit into it; I tried to simplify the code, but it is too entangled,
too many crossreferences; so instead I conjured a small script that only logs
data. Note that the station type has to be configured by editing the initial
part.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1517 root 30 10 18320 1948 780 R 7.5 15.4 0:34.15 wfdriver_light
Original comment by mennu...@debian.org
on 27 Apr 2013 at 3:00
Attachments:
hi
there is a fundamental problem in Python
http://bugs.python.org/issue9072
"Please accept that Python indeed does not support unloading modules for
severe, fundamental, insurmountable, technical problems, in 2.x."
So the more modules a program uses (even to just call a single method once),
the larger its footprint
If you wish, take a loot into the 'lessmemory' branch
git.debian.org/git/collab-maint/wfrog.git
where I lost some time to no result
I started understanding the problem using the attached snippet.
Still something may be done. IMHO wfrog uses too many threads , and one thread
just contains a time.sleep, while others just do message passing. Refactoring
the code in wflogger.py may improve things a bit (threads are rumored to use
~8MB memory each). my code example wfdriver_light shows that actually the
logging of data may be done w/o any threads.
a.
Original comment by mennu...@debian.org
on 1 May 2013 at 7:24
Attachments:
Original issue reported on code.google.com by
mennu...@debian.org
on 29 Mar 2013 at 6:53