search

cyang-kth / fmm

Fast map matching, an open source framework in C++
https://fmm-wiki.github.io/
Apache License 2.0
876 stars 206 forks source link

Multi-threading support for Python bindings. #218

Open edward17829991 opened 2 years ago

edward17829991 commented 2 years ago

Is your feature request related to a problem? Please describe. Recently, I'm trying out the fantastic FMM module with its Python API, it has been doing great until I want to parallelize it. I want to integrate FMM as part of an REST API server, and as a result, there is only single thread working at a time. After some investigation, I found out that the python binding created by SWIG is not support threading by default. (due to python GIL design) Is there any suggestion on how to make the python binding support threading? I'm not that familiar with SWIG framework actually, so I'm seeking for suggestions.

Describe the solution you'd like In my use cases, "fmm_model.match_wkt" is called in a flask response callback, and the flask app is served by "gunicorn". I've tried to set the parallel workers of gunicorn to greater than one, this worked fine as expected. But the ubodt file is loaded per process which is very memory consuming. When I tried to set the parallel threads of gunicorn to greater than one, only one thread was working at a time.

Describe alternatives you've considered I am thinking is there a way of configuring the SWIG for supporting threading?

Many thanks in advance.

Best, Edward Chen

cyang-kth commented 2 years ago

Sorry, I did not have experience with this part. The parallelization of FMM is achieved with OpenMP but I don't think that is what you want. Perhaps you need to search for other resources.