Flex: Error handling w/ LuaJIT

[mmd-osm]

Errors during processing trigger a dump when using LuaJIT. I'm suspecting that LuaJIT somehow chokes during stack unwinding a C++ -> Lua -> C++ stack. The issue is LuaJIT specific, and doesn't occur on Lua 5.2.

Platform: Ubuntu 18.04, x64

osm2pgsql version 1.2.0 (1.2.0-246-g8b3c3cb) (64 bit id space)

Compiled using the following library versions:
Libosmium 2.15.4
Lua 5.1.4 (LuaJIT 2.1.0-beta3)

./osm2pgsql -d osm2 --create --slim -P 5434  -O flex -S ../flex-config/default-config.lua saarland-latest.osm.pbf 
osm2pgsql version 1.2.0 (1.2.0-246-g8b3c3cb) (64 bit id space)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=11
Mid: pgsql, cache=800
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using projection SRS 3857 (Spherical Mercator)

Reading in file: saarland-latest.osm.pbf
Using PBF parser.
Processing: Node(3652k 521.7k/s) Way(624k 89.14k/s) Relation(0 0.00/s)terminate called after throwing an instance of 'std::runtime_error'
  what():  Missing geometry transformation for column 'geom'
Aborted (core dumped)

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff5d1d801 in __GI_abort () at abort.c:79
#2  0x00007ffff6710957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff6716ab6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff6716af1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff67164ba in __gxx_personality_v0 () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff60de613 in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#7  0x00007ffff60deb71 in _Unwind_RaiseException () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#8  0x00007ffff6a22c9b in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#9  0x00007ffff6a375af in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#10 0x00007ffff6a37637 in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#11 0x00007ffff6a4aa29 in luaL_error () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#12 0x0000555555605e36 in lua_trampoline_table_add_row (lua_state=0x40000378) at /home/dir/osm2pgsql/src/output-flex.cpp:64
#13 0x00007ffff6a15e37 in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#14 0x00007ffff6a6327c in lua_pcall () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
#15 0x0000555555602473 in output_flex_t::call_process_function (this=this@entry=0x5555558b6cf0, index=index@entry=3, object=...) at /home/dir/osm2pgsql/src/output-flex.cpp:838
#16 0x0000555555602c88 in output_flex_t::relation_add (this=0x5555558b6cf0, relation=...) at /home/dir/osm2pgsql/src/output-flex.cpp:1039
#17 0x00005555555c9cc0 in osmdata_t::relation_add (this=<optimised out>, rel=...) at /home/dir/osm2pgsql/src/osmdata.cpp:65
#18 0x00005555555de4c5 in parse_osmium_t::relation (this=this@entry=0x7fffffffd520, rel=...) at /home/dir/osm2pgsql/src/parse-osmium.cpp:223
#19 0x00005555555df1e0 in osmium::detail::apply_item_impl<parse_osmium_t&, osmium::memory::Item> (handler=..., item=...)
    at /home/dir/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:69
#20 osmium::apply_item<osmium::memory::Item, parse_osmium_t&> (item=...) at /home/dir/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:303
#21 osmium::apply_impl<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, parse_osmium_t&> (end=..., it=...)
    at /home/dir/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:317
#22 osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, parse_osmium_t&> (end=..., it=...)
    at /home/dir/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:324
#23 osmium::apply<osmium::io::Reader, parse_osmium_t&> (c=...) at /home/dir/osm2pgsql/contrib/libosmium/osmium/visitor.hpp:331
#24 parse_osmium_t::stream_file (this=0x7fffffffd520, filename=..., fmt="auto") at /home/dir/osm2pgsql/src/parse-osmium.cpp:148
#25 0x00005555555a8d9b in main (argc=<optimised out>, argv=<optimised out>) at /home/dir/osm2pgsql/src/osm2pgsql.cpp:96

Lua 5.2 would report: _Osm2pgsql failed due to ERROR: Failed to execute lua processing function: ../flex-config/default-config.lua:245: Error in 'addrow': Missing geometry transformation for column 'geom'

[joto]

The C++ code catches all exceptions and calls luaL_error(), so there is no exception crossing the C++/Lua boundary. I am not sure exectly what's going on here, but removing the noexcept from the function called by Lua fixes the problem for me. @mmd-osm Please test whether the PR works for you.

[mmd-osm]

Retest looks good here!

./osm2pgsql -d osm2 --create --slim -P 5434  -O flex -S ../flex-config/default-config.lua saarland-latest.osm.pbf 
osm2pgsql version 1.2.0 (1.2.0-246-g8b3c3cb-changed) (64 bit id space)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=11
Mid: pgsql, cache=800
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using projection SRS 3857 (Spherical Mercator)

Reading in file: saarland-latest.osm.pbf
Using PBF parser.
Processing: Node(3652k 521.7k/s) Way(598k 85.43k/s) Relation(0 0.00/s)node cache: stored: 3652013(100.00%), storage efficiency: 50.08% (dense blocks: 15, sparse nodes: 3584640), hit rate: 100.00%
Osm2pgsql failed due to ERROR: Failed to execute lua processing function: ../flex-config/default-config.lua:245: Error in 'add_row': Missing geometry transformation for column 'geom'

./osm2pgsql -V
osm2pgsql version 1.2.0 (1.2.0-246-g8b3c3cb-changed) (64 bit id space)

Compiled using the following library versions:
Libosmium 2.15.4
Lua 5.1.4 (LuaJIT 2.1.0-beta3)