I'm trying to use python-mapnik and fusepy together in the same project. They work fine when run in separate processes, but hang on a futex wait when run in the same process and mapnik references a fuse path. This appears to be because python threads are not unblocked in create_datasource and other C calls.
As an explicit example, the following will hang waiting for a futex forever:
import fuse
import mapnik
import tempfile
import threading
import time
path = tempfile.mkdtemp()
print('Mount path: %s' % path)
t = threading.Thread(target=fuse.FUSE, args=(fuse.Operations(), path), kwargs={'foreground': True})
t.daemon = True
t.start()
print('Wait a short time for fuse')
time.sleep(1)
print('Ready')
mapnik.Gdal(base=None, file=path)
print('Done')
Note that the python fuse is a no-op, and the path we are opening with gdal should return that it is a directory and can't be opened, but it never gets there.
If I modify src/mapnik_datasource.cpp, including mapnik_threads.hpp, and placing python_unblock_auto_block b; before the return mapnik::datasource_cache::instance().create(params); call in create_datasource, it no longer has this problem -- the above code will fail as expected.
However, I still can't open an actual file with Gdal, which suggests that this needs to be done in other locations.
Is there a way to do this more generally (or a better way to fix this)? If I was sure of all the places where this sort of unblocking was necessary, I could make a PR with the appropriate changes.
I'm trying to use python-mapnik and fusepy together in the same project. They work fine when run in separate processes, but hang on a futex wait when run in the same process and mapnik references a fuse path. This appears to be because python threads are not unblocked in
create_datasource
and other C calls.As an explicit example, the following will hang waiting for a futex forever:
Note that the python fuse is a no-op, and the path we are opening with gdal should return that it is a directory and can't be opened, but it never gets there.
If I modify
src/mapnik_datasource.cpp
, includingmapnik_threads.hpp
, and placingpython_unblock_auto_block b;
before thereturn mapnik::datasource_cache::instance().create(params);
call increate_datasource
, it no longer has this problem -- the above code will fail as expected.However, I still can't open an actual file with Gdal, which suggests that this needs to be done in other locations.
Is there a way to do this more generally (or a better way to fix this)? If I was sure of all the places where this sort of unblocking was necessary, I could make a PR with the appropriate changes.