search

somnathrakshit / geograpy3

Extract place names from a URL or text, and add context to those names -- for example distinguishing between a country, region or city.
https://geograpy3.readthedocs.io
Apache License 2.0
124 stars 12 forks source link

[BUG] location.db access fails within read-only docker container #58

Closed DanielHabenicht closed 1 year ago

DanielHabenicht commented 3 years ago

Describe the bug Geograpy fails with the following error if used on read-only docker containers: 

Traceback (most recent call last):
  File "/app/index.py", line 152, in <module>
    places = geograpy.get_geoPlace_context(
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/__init__.py", line 24, in get_geoPlace_context
    places=get_place_context(url, text, labels=Labels.geo, debug=debug)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/__init__.py", line 46, in get_place_context
    pc = PlaceContext(e.places)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/places.py", line 29, in __init__
    super().__init__()
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/locator.py", line 184, in __init__
    self.sqlDB=SQLDB(self.db_file,errorDebug=True)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/lodstorage/sql.py", line 41, in __init__
    self.c=sqlite3.connect(dbname,detect_types=sqlite3.PARSE_DECLTYPES,check_same_thread=check_same_thread)
sqlite3.OperationalError: unable to open database file

To Reproduce Use geograpy as part of a script inside of a read-only docker container (but with writable /tmp folder)

import nltk
import geograpy
nltk.downloader.download('maxent_ne_chunker')
nltk.downloader.download('words')
nltk.downloader.download('treebank')
nltk.downloader.download('maxent_treebank_pos_tagger')
nltk.downloader.download('punkt')
# since 2020-09
nltk.downloader.download('averaged_perceptron_tagger')
places = geograpy.get_geoPlace_context(text)

Expected behavior Should not fail or there should be a way to configure the used directories and files.

Environment (please complete the following information):

Additional context Maybe use /tmp for temporary files instead of "random folder". I also tried setting GEOGRAPY_DB=/tmp/loc.db but this fails with: 

Traceback (most recent call last):
  File "/app/index.py", line 152, in <module>
    places = geograpy.get_geoPlace_context(
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/__init__.py", line 24, in get_geoPlace_context
    places=get_place_context(url, text, labels=Labels.geo, debug=debug)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/__init__.py", line 46, in get_place_context
    pc = PlaceContext(places)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/places.py", line 32, in __init__
    self.setAll()
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/places.py", line 87, in setAll
    self.set_countries()
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/places.py", line 98, in set_countries
    country=self.getCountry(place)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/geograpy/locator.py", line 1162, in getCountry
    countryRecords=self.sqlDB.query(query,params)
  File "/home/friendlyusername/.local/lib/python3.9/site-packages/lodstorage/sql.py", line 183, in query
    query = cur.execute(sqlQuery,params)
sqlite3.OperationalError: no such table: countries
WolfgangFahl commented 3 years ago

Thank you for your bug report. The locations.db is expected to be in $HOME/.geograpy3/locations.db and will be automatically downloaded and extracted if it is not there - at least that is the idea of the current design.

For reproducting I'd need to know more about your docker environment/dockerfile. We might also offer and official docker image if there is a request for it.

somnathrakshit commented 2 years ago

Please use the latest version (0.2.3) from PyPi to see if this fixes your error. https://github.com/somnathrakshit/geograpy3/compare/v0.2.2...v0.2.3

WolfgangFahl commented 1 year ago

Please reopen as a feature request if the problem persistst.