Closed yoshi314 closed 3 years ago
lonvia
commented
3 years ago The default settings for the new legacy_icu tokenizer were not being installed, which I have fixed in 2992dea5c8d206cd332c5a5e782f33c497953f4c. but other than that I can't reproduce this behaviour.
It sounds like you have a half-installed version of Nominatim. Try reinstalling a clean version of Nominatim and make sure your project directory is well separated from the Nominatim source, build and installation directory.
yoshi314
commented
3 years ago hmm still hitting this error. Really stumped here as to what could be wrong. Will retry with legacy tokenizer this time.
lonvia
commented
3 years ago Can you describe the exact steps how you installed Nomiantim and how you did the import? Also, there is a CI script successfully doing the import with the legacy tokenizer. Maybe you can check that out and find a step where your method differs.
yoshi314
commented
3 years ago i did a git checkout
mkdir build cd build cmake .. (resolve all misssing deps) make make install
made separate project directory ~/_nominatim with aforementioned env file and tried to import europe merged with canary islands.
This system previously served nominatim 3.4.1 installed in ~/Nominatim/ + ~/Nominatim/build but it should not conflict with this one.
If that might be the case, I'll try to setup a clean system.
lonvia
commented
3 years ago Sounds exactly right. This is odd. The 3.4.1 installation should not interfere.
I suspect something still goes wrong with the installation of the nominatim binary. Can you do an ls /usr/local/lib/nominatim/lib-python/nominatim/tokenizer and let me know the contents of /usr/local/bin/nominatim. Also make sure that which nominiatim really points to the one in /usr/local.
yoshi314
commented
3 years ago tokenizers: (should init be empty?)
-rw-r--r-- 1 root root 3068 May 13 09:04 factory.py
-rw-r--r-- 1 root root 0 May 13 09:04 __init__.py
-rw-r--r-- 1 root root 23767 May 14 10:54 legacy_icu_tokenizer.py
-rw-r--r-- 1 root root 21865 May 14 10:54 legacy_tokenizer.pynominatim script
#!/usr/bin/env python3
import sys
import os
sys.path.insert(1, '/usr/local/lib/nominatim/lib-python')
os.environ['NOMINATIM_NOMINATIM_TOOL'] = os.path.abspath(__file__)
from nominatim import cli
exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
osm2pgsql_path='/usr/local/lib/nominatim/osm2pgsql',
phplib_dir='/usr/local/lib/nominatim/lib-php',
sqllib_dir='/usr/local/lib/nominatim/lib-sql',
data_dir='/usr/local/share/nominatim',
config_dir='/usr/local/etc/nominatim',
phpcgi_path='/usr/bin/php-cgi'))I distinctly recall that when i tried checkout last tag, i had bizarre build errors. Maybe that could help here.
lonvia
commented
3 years ago That' s all as it should be, including the empty __init__.py file (that's just the usual marker for Python to know that this directory contains Python source files).
Hmm, maybe there is another error hidden here. Can you please add a print to /usr/local/lib/nominatim/lib-python/nominatim/tokenizer/factory.py in the except block around line 32. It should then look like this:
try:
import sys
print(sys.path)
return importlib.import_module('nominatim.tokenizer.' + name + '_tokenizer')
except ModuleNotFoundError as exp:
print("Exception", exp) # <------ add this line here
LOG.fatal("No tokenizer named '%s' available. "
"Check the setting of NOMINATIM_TOKENIZER.", name)
raise UsageError('Tokenizer not found') from exp(If you are not familiar with Python: careful with the spaces in front of the line. Make sure there are exactly as many as on the line below and no Tabs.)
yoshi314
commented
3 years ago With legacy_icu go i got
Module 'icu' not found. So i suppose missing dependencies on my end. I wonder what's breaking the legacy one.
lonvia
commented
3 years ago Aha. It's really bad that the error message hides the underlying issue. I shall fix that.
You need the python ICU library: apt install python3-icu. The development docs should always have the full dependency list for master: https://nominatim.org/release-docs/develop/appendix/Install-on-Ubuntu-20/
yoshi314
commented
3 years ago It seems to be going now, but yes - a dependency check would be nice to have before the entire loading process begins.
During initial import of the osm data, i get an error
Same thing for legacy tokenizer.
To Reproduce Build current git on debian10. Try to import Europe dump.
Software Environment (please complete the following information):
Hardware Configuration (please complete the following information):
My .env :
i also tried legacy value for tokenizer, with the exact same error message.