search

petewarden / dstk

A collection of the best open data sets and open-source tools for data science
http://www.datasciencetoolkit.org/
1.13k stars 184 forks source link

street2coordinates fails after about 1000 queries #4

Closed petewarden closed 13 years ago

petewarden commented 13 years ago

From email:

I'm running DSTK on an EC2 instance, and after roughly 1000 queries to street2coordinates/ I get the following error. The error appears no matter what street address is given. Rebooting the EC2 image fixes it, but it reappears predictably after another 1000 or so queries. Is it possible there query limits on the EC2 image? If so, how do I remove them?

{"error":"street2coordinates error: #[\"/usr/lib/ruby/1.8/sqlite3/ errors.rb:62:in check'\", \"/usr/lib/ruby/1.8/sqlite3/database.rb: 79:ininitialize'\", \"../geocoder/lib/geocoder/us/database.rb:38:in new'\", \"../geocoder/lib/geocoder/us/database.rb:38:in initialize'\", \"./dstk_server.rb:457:in new'\", \"./dstk_server.rb: 457:instreet2coordinates'\", \"./dstk_server.rb:953:in `GET / street2coordinates/*'\", ...

petewarden commented 13 years ago

Created a test case as bug_0004.py

This was caused by unused geocoder database objects not being garbage-collected. I fixed this by making a global singleton to hold the object. Should be in version 0.40.