search

jannikmi / timezonefinder

python package for finding the timezone of any point on earth (coordinates) offline
https://timezonefinder.michelfe.it/
MIT License
450 stars 48 forks source link

Too many open files: '/usr/local/lib/python3.6/site-packages/timezonefinder/poly_zone_ids.bin' #57

Closed tristaneljed closed 6 years ago

tristaneljed commented 6 years ago

I am using this library within my python app hosted in a docker image, and when I started using it it was okay, and now I'm having this issue. My guess is that somewhere in the code the object is not being garbage collected/destroyed and the binaries remain open over time causing the issue. Any inputs?

Unable to find source-code formatter for language: text. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.6/site-packages/timezonefinder/poly_zone_ids.bin' 172.17.0.21 - - [08/May/2018 07:10:23] "POST /1.0/predict HTTP/1.1" 500 - Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1997, in call

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1985, in wsgi_app

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1540, in handle_exception

File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception

File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request

File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request

File "/usr/src/app/app.py", line 141, in is_at_venue

File "/usr/src/app/utils/AtVenueModel.py", line 38, in init

File "/usr/src/app/utils/TemporalTools.py", line 12, in getLocalTime

File "/usr/local/lib/python3.6/site-packages/timezonefinder/timezonefinder.py", line 104, in init

adamchainz commented 6 years ago

This is likely to do with your app or some other package, timezonefinder doesn't open that many files. In unix, every process has a limit on the number of files it can open. You might be opening files and not closing them?

tristaneljed commented 6 years ago

Not really, it's a simple flask API. I'll investigate more into the code of timezonefinder and see the cause of the issue.

adamchainz commented 6 years ago

If you use open not as a context manager, i.e. foo = open('filename'), that creates a new file handle. Unless foo gets garbage collected, or foo.close() is called, the file handle will remain open.

If you run strace python myapp.py you will see all the calls to open, that might help debug what's opening so many files.

jannikmi commented 6 years ago

FYI: at the moment the TimezoneFinder class keeps 14 files open until it gets deleted (the del function is being called). This is intended behavior.