"Unknown index type" for an available index types

[Danysan1]

What version of osmium-tool are you using?

# osmium --version
osmium version 1.10.0
libosmium version 2.15.0

What operating system version are you using?

# uname -a
Linux 88de86326fdd 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 GNU/Linux

# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian

Tell us something about your system

Running inside a Docker container based on the target dev of this Dockerfile based on Debian Buster.

What did you do exactly?

I read the documentation here:

When using the file-based index types (*_file_array), add the filename you want to use for the index after a comma to the index types like so:

... -i dense_file_array,index.dat ...

From this script I ran:

# osmium export --verbose --overwrite -o '/var/www/html/filtered_extract.osm.pbf.pg' -f 'pg' --config='osmium.json' --add-unique-id='counter' --index-type=dense_file_array,/tmp/osmium-nodes.cache '/var/www/html/filtered_extract.osm.pbf'
Unknown index type 'dense_file_array,/tmp/osmium-nodes.cache'. Use --show-index-types or -I to get a list.

These are the available index types:

# osmium export --show-index-types
dense_file_array
dense_mem_array
dense_mmap_array
flex_mem
sparse_file_array
sparse_mem_array
sparse_mem_map
sparse_mmap_array
none

What did you expect to happen?

I expected osmium export to execute using dense_file_array indexing

What did happen instead?

I receive an error such as Unknown index type 'dense_file_array,/tmp/osmium-nodes.cache'. Use --show-index-types or -I to get a list.

What did you do to try analyzing the problem?

I tried changing --index-type=dense_file_array,/tmp/osmium-nodes.cache to --index-type='dense_file_array,/tmp/osmium-nodes.cache' :

# osmium export --verbose --overwrite -o '/var/www/html/filtered_extract.osm.pbf.pg' -f 'pg' --config='osmium.json' --add-unique-id='counter' --index-type='dense_file_array,/tmp/osmium-nodes.cache' '/var/www/html/filtered_extract.osm.pbf'
Unknown index type 'dense_file_array,/tmp/osmium-nodes.cache'. Use --show-index-types or -I to get a list.

I also tried changing --index-type=dense_file_array,/tmp/osmium-nodes.cache to --index-type=dense_file_array,osmium-nodes.cache :

# osmium export --verbose --overwrite -o '/var/www/html/filtered_extract.osm.pbf.pg' -f 'pg' --config='osmium.json' --add-unique-id='counter' --index-type=dense_file_array,osmium-nodes.cache '/var/www/html/filtered_extract.osm.pbf'
Unknown index type 'dense_file_array,osmium-nodes.cache'. Use --show-index-types or -I to get a list.

I also tried changing --index-type=dense_file_array,/tmp/osmium-nodes.cache to --index-type=sparse_file_array,/tmp/osmium-nodes.cache:

# osmium export --verbose --overwrite -o '/var/www/html/filtered_extract.osm.pbf.pg' -f 'pg' --config='osmium.json' --add-unique-id='counter' --index-type=sparse_file_array,/tmp/osmium-nodes.cache '/var/www/html/filtered_extract.osm.pbf'
Unknown index type 'sparse_file_array,/tmp/osmium-nodes.cache'. Use --show-index-types or -I to get a list.

I am running as root and I have accesso to /tmp. /tmp/osmium-nodes.cache does not exist at this time.

Without any --index-type it works fine for a small file, but I already know that on my setup I won't be able to handle a planet file, which I will need to use sooner or later:

# osmium export --verbose --overwrite -o '/var/www/html/filtered_extract.osm.pbf.pg' -f 'pg' --config='osmium.json' --add-unique-id='counter' '/var/www/html/filtered_extract.osm.pbf'[ 0:00] Started osmium export
[ 0:00]   osmium version 1.10.0
[ 0:00]   libosmium version 2.15.0
[ 0:00] Command line options and default settings:
[ 0:00]   input options:
[ 0:00]     file name: /var/www/html/filtered_extract.osm.pbf
[ 0:00]     file format:
[ 0:00]   output options:
[ 0:00]     file name: /var/www/html/filtered_extract.osm.pbf.pg
[ 0:00]     file format: pg
[ 0:00]     overwrite: yes
[ 0:00]     fsync: no
[ 0:00]   attributes:
[ 0:00]     type:      @type
[ 0:00]     id:        @id
[ 0:00]     version:   (omitted)
[ 0:00]     changeset: (omitted)
[ 0:00]     timestamp: (omitted)
[ 0:00]     uid:       (omitted)
[ 0:00]     user:      (omitted)
[ 0:00]     way_nodes: (omitted)
[ 0:00]   linear tags: any
[ 0:00]   area tags:   any
[ 0:00]   include only these tags:
[ 0:00]     name
[ 0:00]     wikidata
[ 0:00]     subject:wikidata
[ 0:00]     name:etymology:wikidata
[ 0:00]   other options:
[ 0:00]     index type: flex_mem
[ 0:00]     add unique IDs: counter
[ 0:00]     keep untagged features: no
[ 0:00] First pass (of two) through input file (reading relations)...
[ 0:00] First pass done.
[ 0:00] Second pass (of two) through input file...
[ 0:00] 
[ 0:00] Create table with something like this:
[ 0:00] CREATE TABLE osmdata (
[ 0:00]     id        BIGINT PRIMARY KEY,
[ 0:00]     geom      GEOMETRY,
[ 0:00]     osm_type  VARCHAR,
[ 0:00]     osm_id    BIGINT,
[ 0:00]     tags      JSON -- or JSONB
[ 0:00] );
[ 0:00] Then load data with something like this:
[ 0:00] \copy osmdata FROM '/var/www/html/filtered_extract.osm.pbf.pg'
[ 0:00]
[======================================================================] 100% 
[ 0:04] About 30 MBytes used for node location index (in main memory or on disk).
[ 0:04] Second pass done.
[ 0:04] Wrote 22194 features.
[ 0:04] Encountered 0 errors.
[ 0:04] Peak memory used: 1028 MBytes
[ 0:04] Done.

[joto]

This is a known bug. It was fixed in Osmium 1.12.0. Please upgrade.

[Danysan1]

After upgrading to Debian bullseye and osmose 1.13.1 it works. Thank you.