digideskio / googletransitdatafeed

Automatically exported from code.google.com/p/googletransitdatafeed
0 stars 0 forks source link

Memory Leak in transitfeed Python module #354

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
The following code results in a memory leak in the transitfeed library when 
processing multiple GTFS feeds (200+, but it's noticeable after just a few) in 
a loop.  A full script to exhibit the behavior is attached.

  for f in os.listdir("./gtfs/"):
    loader = transitfeed.Loader(feed_path="./gtfs/"+f, memory_db=False)
    schedule = loader.Load()    
    #del schedule
    #del loader

The following things have been attempted unsuccessfully to eliminate the leak:
1. Explicitly "del"-ing the objects (uncomment above code).
2. Creating the loader object with "memory_db=False"
3. Using the "map" function instead of a loop.

Since a schedule object creates a sqlite3 database and connection (in memory or 
a tempfile), my guess is that the connection is not being closed properly.  
However, the __del__ method for Schedule does seem to attempt to do this...

What version of the product are you using? On what operating system?
- I have tested this with transitfeed 1.2.12 (both from SVN and PIP).
- I am using Ubuntu 12.10 64-bit with Python 2.7.3 and libsqlite3 3.7.13.

Original issue reported on code.google.com by liquidsu...@gmail.com on 30 Nov 2012 at 5:48

Attachments:

GoogleCodeExporter commented 9 years ago

Original comment by bdfer...@google.com on 26 Sep 2014 at 4:51

GoogleCodeExporter commented 9 years ago
Moved to https://github.com/google/transitfeed/issues/354

Original comment by bdfer...@google.com on 7 Oct 2014 at 7:41