robjuz / helm-charts

https://robjuz.github.io/helm-charts/index.yaml
34 stars 30 forks source link

subprocess.CalledProcessError: Command died with <Signals.SIGKILL: 9> #57

Open felipemendes1994 opened 1 year ago

felipemendes1994 commented 1 year ago

Hello,

I've been trying to run nominatim pointing to a global map, through Google Cloud infra with helm, and I've been facing the following error.

Defaulted container "nominatim" out of: nominatim, wait-for-db (init), download-pbf (init)
2023-04-25 17:15:36: Using project directory: /nominatim
2023-04-25 17:15:38: Creating database
2023-04-25 17:15:39: Setting up country tables
2023-04-25 17:15:41: Importing OSM data file
2023-04-25 17:15:42  osm2pgsql version 1.7.1
2023-04-25 17:15:42  Database version: 14.5
2023-04-25 17:15:42  PostGIS version: 3.1
2023-04-25 17:15:42  Parsing gazetteer style file '/usr/local/etc/nominatim/import-full.style'.
NOTICE:  table "place" does not exist, skipping
Processing: Node(2129400k 676.9k/s) Way(0k 0.00k/s) Relation(0 0.0/s)Traceback (most recent call last):
  File "/usr/local/bin/nominatim", line 14, in <module>
    exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
  File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 264, in nominatim
    return parser.run(**kwargs)
  File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 126, in run
    return args.command.run(args)
  File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/setup.py", line 92, in run
    database_import.import_osm_data(files,
  File "/usr/local/lib/nominatim/lib-python/nominatim/tools/database_import.py", line 108, in import_osm_data
    run_osm2pgsql(options)
  File "/usr/local/lib/nominatim/lib-python/nominatim/tools/exec_utils.py", line 152, in run_osm2pgsql
    subprocess.run(cmd, cwd=options.get('cwd', '.'),
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/lib/nominatim/osm2pgsql', '--hstore', '--latlon', '--slim', '--with-forward-dependencies', 'false', '--log-progress', 'true', '--number-processes', '1', '--cache', '32621', '--output', 'gazetteer', '--style', '/usr/local/etc/nominatim/import-full.style', '--create', 'data.osm.pbf']' died with <Signals.SIGKILL: 9>.

I saw on other issues this is related to out of memory problems, but It's kinda weird cause on cloud infrastructure, we have "unlimited" resources.

Troubleshooting the received error, I noticed neither on database, neither on pod, nominatim ran out of resources (vCPUs and Memory).

For my pod (nominatim-init), I have the following usage: Captura de tela de 2023-04-25 17-05-29

When nominatim-init pod was created was allocated to a e2-highcpu-32 node pool (31.85 CPU 29.78 GB, Allocable). And as we can see, the resource usage, didn't get close to the limit.


My database instance is running on Cloud SQL, with the following configuration: vCPUs: 24 Memory: 80 GB Storage: 100GB SSD Initialy (Auto Increase enabled).

And according to the following images, didn't get close to the limits also. Captura de tela de 2023-04-25 17-05-58

Captura de tela de 2023-04-25 17-09-49

Captura de tela de 2023-04-25 17-22-28


Can anyone tell what's going wrong and how can I solve it?

alterEgo123 commented 5 months ago

Hello, did you manage to solve this?