Memory leak when repeatedly observing satellite

[fwtu]

Working with skyfield I observed a possible memory leak when one repeatedly calculates a satellite's altitude like in the satellite observation notebook. Feel free to close this issue if I'm doing sth. wrong in the script below.

Steps to reproduce:

1. mkvirtualenv skyfield-test; pip install skyfield
2. Run ./test.py attached below
3. Watch the memory usage grow
4. Hit Ctrl+C and see the output of tracemalloc

The first line of the output (highest memory usage) will be for instance: [...]/skyfield/timelib.py:602: size=6962 KiB, count=52419, average=136 B This indicates a leak of def delta_t() in the following code:

def interpolate_delta_t(cache, tt):
    """Given TT, return interpolated Delta T, falling back to a formula."""
    def delta_t():  # TODO
        "Fake placeholder function, until I rewrite how the cache works."
    x, y = cache.run(delta_t)
    delta_t = interp(tt, x, y, nan, nan)
    missing = isnan(delta_t)
    if missing.any():
        tt = tt[missing]
        delta_t[missing] = delta_t_formula_morrison_and_stephenson_2004(tt)
    return delta_t

When replacing the complete function body as shown below, the memory usage is stable.

def interpolate_delta_t(cache, tt):
    return delta_t_formula_morrison_and_stephenson_2004(tt)

test.py script:

#!/usr/bin/env python3

import tracemalloc
from skyfield.api import load, JulianDate, Angle

tle = """ISS (ZARYA)
1 25544U 98067A   16036.52823010  .00009121  00000-0  14318-3 0  9990
2 25544  51.6441 359.5913 0006959  83.4035  25.9071 15.54440498984346
"""
lat = 50.000
lon = 10.000
starttime = (2016, 2, 11, 6, 0)

planets = load("de421.bsp")
earth = planets["earth"]
topos = earth.topos(Angle(degrees=lat), Angle(degrees=lon))
sat = earth.satellite(tle)
t = JulianDate(utc=starttime).tai
t_step = 1.0 / 24 / 60 / 60

tracemalloc.start()
try:
    while True:
        t += t_step
        alt = topos.at(tai=t).observe(sat).altaz()[0].degrees
except KeyboardInterrupt:
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')
    for stat in top_stats[:10]:
        print(stat)

[brandon-rhodes]

Could you try this test again with the most recent version of the code, and see whether you get a better result? The inner function has been removed as part of several cleanup steps, and — happily — I am not able to reproduce the memory problem you see. Let me know if you see the same. Thanks!

[fwtu]

Just tried to modify my test script to work with the new version and tested it again. However, it crashes with the following exception:

(skyfield-test) [fw@zen ~]$ python -V
Python 3.5.1
(skyfield-test) [fw@zen ~]$ pip freeze
jplephem==2.5
numpy==1.11.0
sgp4==1.4
skyfield==0.8
(skyfield-test) [fw@zen ~]$ ./test.py 
Traceback (most recent call last):
  File "./test.py", line 26, in <module>
    alt = topos.at(ts.tai(t)).observe(sat).altaz()[0].degrees
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/errors.py", line 37, in wrapper
    return method(self, jd)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/toposlib.py", line 57, in at
    tpos_au, tvel_au_per_d = self._position_and_velocity(t)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/toposlib.py", line 76, in _position_and_velocity
    self.elevation.au, t.gast)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/timelib.py", line 510, in __getattr__
    self.gast = gast = self.gmst + earth_tilt(self)[2] / 3600.0
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/timelib.py", line 506, in __getattr__
    self.gmst = gmst = sidereal_time(self)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/earthlib.py", line 107, in sidereal_time
    theta = earth_rotation_angle(t.ut1)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/timelib.py", line 497, in __getattr__
    self.ut1 = ut1 = self.tt - self.delta_t / DAY_S
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/timelib.py", line 502, in __getattr__
    self.delta_t = delta_t = interpolate_delta_t(table, self.tt)
  File "/home/fw/.virtualenvs/skyfield-test/lib/python3.5/site-packages/skyfield/timelib.py", line 603, in interpolate_delta_t
    delta_t[missing] = delta_t_formula_morrison_and_stephenson_2004(tt)
TypeError: 'float' object does not support item assignment

Script is:

#!/usr/bin/env python3

import tracemalloc
from skyfield.api import load, Angle

tle = """ISS (ZARYA)
1 25544U 98067A   16036.52823010  .00009121  00000-0  14318-3 0  9990
2 25544  51.6441 359.5913 0006959  83.4035  25.9071 15.54440498984346
"""
lat = 50.000
lon = 10.000
starttime = [2016, 2, 11, 6, 0]

planets = load("de421.bsp")
earth = planets["earth"]
topos = earth.topos(Angle(degrees=lat), Angle(degrees=lon))
sat = earth.satellite(tle)
ts = load.timescale()
t = ts.utc(*starttime).tai
t_step = 1.0 / 24 / 60 / 60

tracemalloc.start()
try:
    while True:
        t += t_step
        alt = topos.at(ts.tai(t)).observe(sat).altaz()[0].degrees
except KeyboardInterrupt:
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')
    for stat in top_stats[:10]:
        print(stat)

Also tested it with the current code from git. Am I doing something wrong?

[brandon-rhodes]

The problem you ran into has now been resolved! I will be releasing a new 0.9 version of Skyfield later this weekend, which should resolve the exception you got and let us (finally) make more progress on this issue that you ran into.

[fwtu]

Sorry for the late response - I did another test with skyfield 1.0 and the issue seems to be resolved now, at least when using datetime.

Updated testcase:

#!/usr/bin/env python3

import os
import time
import datetime
import tracemalloc
from skyfield.api import load, Angle, Topos, EarthSatellite, utc

tle = """ISS (ZARYA)
1 25544U 98067A   17112.88405880  .00002035  00000-0  37993-4 0  9998
2 25544  51.6428 313.9569 0007141  83.9098  53.2400 15.54144987 53150
"""
lat = 50.000
lon = 10.000

planets = load("de421.bsp")
earth = planets["earth"]
topos = earth + Topos(Angle(degrees=lat), Angle(degrees=lon))
line1, line2 = tle.splitlines()[-2:]
sat = earth + EarthSatellite(line1, line2)
ts = load.timescale()

t = time.time()
tracemalloc.start()
try:
    while True:
        t += 1.
        observed = topos.at(
            ts.utc(datetime.datetime.fromtimestamp(t, tz=utc))
        ).observe(sat).apparent().altaz()[0].degrees
except KeyboardInterrupt:
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')
    for stat in top_stats[:10]:
        print(str(stat).replace(os.environ.get("VIRTUAL_ENV", "XXX"), "[ENV]"))

Results on my machine for running it for 10 and 20 seconds:

$ timeout -s INT 10 python skyfield_1_0_test.py
[ENV]/lib/python3.6/site-packages/skyfield/jpllib.py:159: size=80.4 KiB, count=1471, average=56 B
[ENV]/lib/python3.6/site-packages/jplephem/spk.py:118: size=77.3 KiB, count=1227, average=65 B
[ENV]/lib/python3.6/site-packages/skyfield/jpllib.py:158: size=4968 B, count=69, average=72 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:482: size=1848 B, count=4, average=462 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:520: size=1800 B, count=5, average=360 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:507: size=1800 B, count=5, average=360 B
[ENV]/lib/python3.6/site-packages/skyfield/vectorlib.py:184: size=1536 B, count=4, average=384 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:466: size=1232 B, count=3, average=411 B
[ENV]/lib/python3.6/site-packages/skyfield/earthlib.py:107: size=1168 B, count=2, average=584 B
[ENV]/lib/python3.6/site-packages/skyfield/vectorlib.py:222: size=1136 B, count=2, average=568 B
timeout -s INT 10 python skyfield_1_0_test.py  13,71s user 25,36s system 388% cpu 10,049 total

$ timeout -s INT 20 python skyfield_1_0_test.py
[ENV]/lib/python3.6/site-packages/skyfield/jpllib.py:159: size=80.5 KiB, count=1472, average=56 B
[ENV]/lib/python3.6/site-packages/jplephem/spk.py:118: size=77.4 KiB, count=1228, average=65 B
[ENV]/lib/python3.6/site-packages/skyfield/jpllib.py:158: size=5184 B, count=72, average=72 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:482: size=1784 B, count=3, average=595 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:520: size=1776 B, count=4, average=444 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:507: size=1776 B, count=4, average=444 B
[ENV]/lib/python3.6/site-packages/skyfield/vectorlib.py:184: size=1472 B, count=3, average=491 B
[ENV]/lib/python3.6/site-packages/skyfield/timelib.py:466: size=1168 B, count=2, average=584 B
[ENV]/lib/python3.6/site-packages/skyfield/earthlib.py:107: size=1168 B, count=2, average=584 B
[ENV]/lib/python3.6/site-packages/skyfield/vectorlib.py:222: size=1136 B, count=2, average=568 B
timeout -s INT 20 python skyfield_1_0_test.py  28,15s user 51,66s system 398% cpu 20,043 total

Thus, I think we can close this issue.

[brandon-rhodes]

Thank you for returning with a report! I'll close the issue, then.