zepheira / amara

Amara 2.0. Amara XML toolkit is an open-source collection of Python tools for XML processing, not just tools that happen to be written in Python, but tools built from the ground up to use Python idioms and take advantage of the many advantages of Python over other programming languages.
http://wiki.xml3k.org/Amara2
Apache License 2.0
23 stars 9 forks source link

structwriter memory leak (originally reported by Chimezie) #17

Open uogbuji opened 11 years ago

uogbuji commented 11 years ago

See: http://groups.google.com/group/akara/browse_thread/thread/61c3dfdee4e9301b

An example template that causes the problem (with a sequence of very large files) is below. It uses pushtree to stream elements to structwriter

def main(..etc..): ..snip.. w = structwriter(indent=u"yes",stream=someStream) feed = w.cofeed( ROOT( E_CURSOR( (RDF.RDFNS, u'rdf:RDF')))) handler = Handle(feed,..etc..) @coroutine def receive_nodes(..etc..): while True: node = yield handler.handle(node..,..etc..) return target = receive_nodes(..etc..) pushtree(doc, u'MedlineCitation', target.send, entity_factory=entity_base) handler.feed.close() target.close() return someStream.getvalue()