pytroll / pytroll-pps-runner

Pytroll runner for PPS
GNU General Public License v3.0
1 stars 8 forks source link

Fix memory leak #53

Closed ninahakansson closed 1 year ago

ninahakansson commented 1 year ago

Replaces the ThreadPool with a simple non parallel Process. We saw that the memory usage was increasing over time. Also the ThreadPool were never sent more than one message at a time.

codecov[bot] commented 1 year ago

Codecov Report

Merging #53 (8c7d1e3) into main (6b2d31d) will increase coverage by 0.03%. The diff coverage is 95.74%.

@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
+ Coverage   66.36%   66.39%   +0.03%     
==========================================
  Files          15       15              
  Lines        2066     2074       +8     
  Branches      297      292       -5     
==========================================
+ Hits         1371     1377       +6     
- Misses        656      658       +2     
  Partials       39       39              
Flag Coverage Δ
unittests 66.39% <95.74%> (+0.03%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
nwcsafpps_runner/tests/test_config.py 100.00% <ø> (ø)
nwcsafpps_runner/l1c_processing.py 74.40% <94.11%> (+1.59%) :arrow_up:
nwcsafpps_runner/tests/test_level1c_runner.py 98.96% <96.66%> (-0.53%) :arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

mraspaud commented 1 year ago

Looks like the tests are failing, is this expected?

adybbroe commented 1 year ago

Tests are most likely failing because Python 3.7 is getting dropped from conda-forge. See #52

ninahakansson commented 1 year ago

Jag håller inte med coveralls om att den raden inte är testad.

TAlonglong commented 1 year ago

Jag håller inte med coveralls om att den raden inte är testad.

OK, I think I see what happens. The fake sleep starts, but the function is killed before the time.sleep finishes and then the coveralls to cover that line.

But I don't see an easy solution

mraspaud commented 1 year ago

Jag håller inte med coveralls om att den raden inte är testad.

OK, I think I see what happens. The fake sleep starts, but the function is killed before the time.sleep finishes and then the coveralls to cover that line.

But I don't see an easy solution

Yes, I agree. An ugly solution would be to replace time.sleep(65) with

for i in range(650):
    time.sleep(0.1)

but I think I'd rather have that line counted as not tested :)