openvenues / libpostal

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
MIT License
4.08k stars 421 forks source link

javacpp-presets-libpostal is thread-safe or not #648

Open shlomishasho27 opened 10 months ago

shlomishasho27 commented 10 months ago

Hey,

I'm utilizing javacpp-presets-libpostal for libpostal version 1.1-1.5.8.

Here's my code implementation:

override def parseAddress(str: String): Task[Map[String, String]] = {
    Resource
        .make(
            Task {
                val options = libpostal_get_address_parser_default_options()
                val address = new BytePointer(str, "UTF-8")
                libpostal_parse_address(address, options)
            }
        ) { response =>
            Task(libpostal_address_parser_response_destroy(response))
        }
        .use { response =>
            Task((0 until response.num_components().toInt).map { i =>
                val label = response.labels(i)
                val component = response.components(i)
                label.getString -> component.getString
            }.toMap)
        }
}

The code runs smoothly in my local environment. However, when deployed to a remote environment under heavy traffic, I encountered the following error:

[error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007f64971c3602]
# An error report file with more information is saved as:

After a brief investigation into previous issues, I came across this and this, but they seem to be older issues. I'd like to clarify whether javacpp-presets-libpostal is thread-safe or not.

shlomishasho27 commented 10 months ago

@albarrentine @saudet FYI

saudet commented 10 months ago

libpostal is not thread-safe, see https://github.com/openvenues/libpostal/issues/641#issuecomment-1695052791