search

geometalab / Vector-Tiles-Reader-QGIS-Plugin

Vector Tiles Reader QGIS-Plugin - QGIS Python plugin which reads Mapbox Vector Tiles from a server, a local MBTiles file or a directory
http://plugins.qgis.org/plugins/vector_tiles_reader/
GNU General Public License v2.0
149 stars 30 forks source link

Error activating plugin (from protobuf) #238

Open barryrowlingson opened 5 years ago

barryrowlingson commented 5 years ago

I can install the plugin but when activated with the checkbox it errors from somewhere in the google protobuf module:

Couldn't load plugin 'vector_tiles_reader' due to an error when calling its classFactory() method 

TypeError: Descriptors should not be created directly, but only retrieved from their parent. 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 335, in startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/__init__.py", line 25, in classFactory
    from .plugin.vtr_plugin import VtrPlugin
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vtr_plugin.py", line 51, in 
    from .vt_reader import VtReader
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vt_reader.py", line 33, in 
    from .util.mp_helper import decode_tile_native, decode_tile_python, load_lib
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/util/mp_helper.py", line 5, in 
    import mapbox_vector_tile
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/__init__.py", line 1, in 
    from . import encoder
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/encoder.py", line 13, in 
    from .compat import PY3, vector_tile, apply_map
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/compat.py", line 7, in 
    from .Mapbox import vector_tile_pb2_p3
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/Mapbox/vector_tile_pb2_p3.py", line 35, in 
    type=None),
  File "/home/rowlings/.local/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 693, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Python version: 3.6.6 (default, Sep 12 2018, 18:26:19) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] 
QGIS version: 3.6.0-Noosa Noosa, 5873452 

Python Path:
/usr/share/qgis/python
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python36.zip
/usr/lib/python3.6
/usr/lib/python3.6/lib-dynload
/home/rowlings/.local/lib/python3.6/site-packages
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python
/home/rowlings/.local/lib/python3.6/site-packages/IPython/extensions
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/home/rowlings/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs
mnboos commented 5 years ago

Thanks for the report!

Looks strange to me. I'll investigate.

In the meantime, would you mind running the following commands in the Python console within QGIS:

from pkgutil import iter_modules
print(sorted([p.name for p in iter_modules()]))
barryrowlingson commented 5 years ago

Here's the modules list:

['AptUrl', 'ClusterPoints', 'CommandNotFound', 'Crypto', 'DistUpgrade', 'HEREqgis', 'HweSupportStatus', 'IPython', 'LanguageSelector', 'MetaSearch', 'NvidiaDetector', 'OSMtools', 'OpenSSL', 'PIL', 'PyQt5', 'QuickOSM', 'Quirks', 'UbuntuDrivers', 'UbuntuSystemService', 'UpdateManager', 'Xlib', 'future', '_asyncio', '_bootlocale', '_bz2', '_cffi_backend', '_codecs_cn', '_codecs_hk', '_codecs_iso2022', '_codecs_jp', '_codecs_kr', '_codecs_tw', '_collections_abc', '_compat_pickle', '_compression', '_crypt', '_csv', '_ctypes', '_ctypes_test', '_curses', '_curses_panel', '_dbm', '_dbus_bindings', '_dbus_glib_bindings', '_decimal', '_dummy_thread', '_gdbm', '_hashlib', '_json', '_lsprof', '_lzma', '_markupbase', '_multibytecodec', '_multiprocessing', '_opcode', '_osx_support', '_pydecimal', '_pyio', '_sitebuiltins', '_smbc', '_sqlite3', '_ssl', '_strptime', '_sysconfigdata_m_linux_x86_64-linux-gnu', '_testbuffer', '_testcapi', '_testimportmultiple', '_testmultiphase', '_threading_local', '_tkinter', '_weakrefset', '_yaml', 'abc', 'absl', 'aifc', 'antigravity', 'apport', 'apport_python_hook', 'apt', 'apt_inst', 'apt_pkg', 'aptdaemon', 'aptsources', 'argh', 'argparse', 'asn1crypto', 'ast', 'astor', 'asynchat', 'asyncio', 'asyncore', 'audioop', 'autoreload', 'babel', 'backcall', 'base64', 'bdb', 'binhex', 'bisect', 'bleach', 'brlapi', 'bs4', 'bz2', 'cProfile', 'cairo', 'calendar', 'certifi', 'cffi', 'cgi', 'cgitb', 'chardet', 'chunk', 'click', 'cmd', 'code', 'codecs', 'codeop', 'collections', 'colorsys', 'compileall', 'concurrent', 'configparser', 'console', 'contextlib', 'copy', 'copyreg', 'crypt', 'cryptography', 'csv', 'ctypes', 'cups', 'cupsext', 'cupshelpers', 'curses', 'cycler', 'cythonmagic', 'datetime', 'dateutil', 'db_manager', 'dbm', 'dbus', 'deb822', 'debconf', 'debian', 'debian_bundle', 'decimal', 'decorator', 'defer', 'defusedxml', 'difflib', 'dis', 'distro_info', 'distro_info_test', 'distutils', 'docs', 'doctest', 'dummy_threading', 'easy_install', 'email', 'encodings', 'entrypoints', 'enum', 'ewmh', 'exifread', 'expressions', 'fail2ban', 'filecmp', 'fileinput', 'first', 'flask', 'flickrapi', 'fnmatch', 'formatter', 'fractions', 'ftplib', 'functools', 'future', 'gast', 'gdal', 'gdalconst', 'gdalnumeric', 'genericpath', 'getopt', 'getpass', 'gettext', 'gi', 'glob', 'gnm', 'google', 'grpc', 'gtweak', 'gzip', 'h5py', 'hashlib', 'heapq', 'hmac', 'hpmudext', 'html', 'html5lib', 'http', 'httplib2', 'idna', 'imaplib', 'imghdr', 'imp', 'importlib', 'inifile', 'inspect', 'io', 'iotop', 'ipaddress', 'ipykernel', 'ipykernel_launcher', 'ipython_genutils', 'ipywidgets', 'itsdangerous', 'janitor', 'jedi', 'jinja2', 'json', 'jsonschema', 'jupyter', 'jupyter_client', 'jupyter_console', 'jupyter_core', 'jupyter_http_over_ws', 'keras', 'keras_applications', 'keras_preprocessing', 'keyring', 'keyword', 'language_support_pkgs', 'launchpadlib', 'lektor', 'lib2to3', 'libexiv2python', 'libfuturize', 'libpasteurize', 'linecache', 'lineedit', 'locale', 'logging', 'louis', 'lsb_release', 'lxml', 'lzma', 'macaroonbakery', 'macpath', 'macurl2path', 'mailbox', 'mailcap', 'mako', 'mapbox_vector_tile', 'markdown', 'markupsafe', 'matplotlib', 'menulibre', 'menulibre_lib', 'mimetypes', 'mistune', 'mmap', 'modulefinder', 'mpl_toolkits', 'multiprocessing', 'nacl', 'nbconvert', 'nbformat', 'netifaces', 'netrc', 'nis', 'nntplib', 'notebook', 'notify2', 'ntpath', 'nturl2path', 'numbers', 'numpy', 'oauth', 'oauthlib', 'ogr', 'olefile', 'opcode', 'operator', 'optparse', 'orca', 'os', 'osgeo', 'osr', 'ossaudiodev', 'owslib', 'pandocfilters', 'parser', 'parso', 'past', 'pathlib', 'pathtools', 'pcardext', 'pdb', 'pexpect', 'pickle', 'pickleshare', 'pickletools', 'pip', 'pipes', 'piptools', 'pkg_resources', 'pkgutil', 'platform', 'plistlib', 'plotly', 'poplib', 'posixpath', 'pprint', 'problem_report', 'processing', 'profile', 'prometheus_client', 'prompt_toolkit', 'pstats', 'psutil', 'psycopg2', 'pty', 'ptyprocess', 'py_compile', 'pyatspi', 'pyclbr', 'pycparser', 'pydoc', 'pydoc_data', 'pyexiv2', 'pygments', 'pygtkcompat', 'pyinotify', 'pylab', 'pymacaroons', 'pyparsing', 'pyplugin_installer', 'pyproj', 'pyrfc3339', 'pytz', 'pyudev', 'qgis', 'qgis2web', 'qrcode', 'qtconsole', 'qtiles3', 'queue', 'quick_map_services', 'quopri', 'radian', 'random', 'rchitect', 're', 'readline', 'reportlab', 'reprlib', 'requests', 'requests_oauthlib', 'requests_toolbelt', 'requests_unixsocket', 'resource', 'rlcompleter', 'rmagic', 'runpy', 'scanext', 'sched', 'scipy', 'scour', 'secrets', 'secretstorage', 'selectors', 'send2trash', 'setproctitle', 'setuptools', 'shapely', 'shelve', 'shlex', 'shutil', 'signal', 'simplegeneric', 'simplejson', 'sip', 'sipconfig', 'sipconfig_nd6', 'site', 'sitecustomize', 'six', 'smbc', 'smtpd', 'smtplib', 'sndhdr', 'socket', 'socketserver', 'softwareproperties', 'speechd', 'speechd_config', 'sqlite3', 'sre_compile', 'sre_constants', 'sre_parse', 'ssh_import_id', 'ssl', 'stat', 'statistics', 'storemagic', 'string', 'stringprep', 'struct', 'subprocess', 'sunau', 'symbol', 'sympyprinting', 'symtable', 'sysconfig', 'systemd', 'tabnanny', 'tarfile', 'taskr', 'telnetlib', 'tempfile', 'tensorboard', 'tensorflow', 'termcolor', 'terminado', 'termios', 'test', 'testpath', 'tests', 'textwrap', 'this', 'threading', 'timeit', 'tkinter', 'token', 'tokenize', 'tornado', 'trace', 'traceback', 'tracemalloc', 'traitlets', 'tty', 'turtle', 'types', 'typing', 'ufw', 'unittest', 'uno', 'unohelper', 'urllib', 'urllib3', 'usbcreator', 'uu', 'uuid', 'vector_tiles_reader', 'venv', 'virtualenv', 'wadllib', 'warnings', 'watchdog', 'wave', 'wcwidth', 'weakref', 'webbrowser', 'webencodings', 'werkzeug', 'wheel', 'widgetsnbextension', 'wsgiref', 'xapp', 'xattr', 'xdg', 'xdrlib', 'xkit', 'xml', 'xmlrpc', 'xxlimited', 'yaml', 'youtube_dl', 'zipapp', 'zipfile', 'zmq', 'zope']

mnboos commented 5 years ago

I hoped we'd see protobuf in there... Hmm..

mnboos commented 5 years ago

Do you see it here:

import site
import os
print([os.listdir(d) for d in site.getsitepackages()])
barryrowlingson commented 5 years ago

That code fails because the first path returned by site.getsitepackages() doesn't exist... but digging in the paths does find a protobuf egg:

>>> site.getsitepackages()
['/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages', '/usr/lib/python3.6/dist-packages']
>>> p = os.listdir('/usr/lib/python3/dist-packages')
>>> [pp for pp in p if pp.startswith("prot")]
['protobuf-3.0.0.egg-info']

Not sure how to get to that... 

>>> import protobuf
Traceback (most recent call last):
  File "/usr/lib/python3.6/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'protobuf'

the google version is this:

>>> from google import protobuf
>>> protobuf.__version__
'3.6.1'
mnboos commented 5 years ago

How about 

pip3 uninstall protobuf

or

pip uninstall protobuf

If this messes up your system, the version you need to reinstall is documented above. 😁

barryrowlingson commented 5 years ago

Fixed! Had to do python3 -m pip uninstall protobuf because pip3 is broken but no problem.

Working now!

mnboos commented 5 years ago

I guess the problem is the following:

There is a protobuf installed in the system, which is found by the plugin. This version is then used instead of the version located in the vecotr_tiles_reader\ext-libs\google directory.

A solution could then be, to make sure this doesn't happen by probably moving them to a VT Reader specific python package.

mnboos commented 5 years ago

I have no idea how to solve this problem. Normally, one can just use a virtual environment, but that's probably not possible.

@sfkeller Would you mind asking one of your Python cracks? Short summary is:

The plugin brings its own version of protobuf, which is registered during startup with site.addsitedir("ext-libs") but in the case above, another, incompatible, version of protobuf was found before and thus the plugin crashed.

sfkeller commented 5 years ago

What's the unique property of plugin's own version of protobuf which differentiates it from the already existant plugin? Just the fact that it's newer? What's the order on how newly registered libs are stored in the list?

CedricLor commented 5 years ago

Hi, I am having the same problem here.

Tried to uninstall protobuf but it did not help.

Couldn't load plugin 'vector_tiles_reader' due to an error when calling its classFactory() method

TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Traceback (most recent call last): File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/__init__.py", line 25, in classFactory from .plugin.vtr_plugin import VtrPlugin File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vtr_plugin.py", line 51, in from .vt_reader import VtReader File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/vt_reader.py", line 33, in from .util.mp_helper import decode_tile_native, decode_tile_python, native_decoding_supported, unload_lib File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/plugin/util/mp_helper.py", line 7, in import mapbox_vector_tile File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/__init__.py", line 1, in from . import encoder File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/encoder.py", line 13, in from .compat import PY3, vector_tile, apply_map File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/compat.py", line 7, in from .Mapbox import vector_tile_pb2_p3 File "/usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 672, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "/Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs/mapbox_vector_tile/Mapbox/vector_tile_pb2_p3.py", line 35, in type=None), File "/usr/local/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 693, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors should not be created directly, but only retrieved from their parent.

Python version: 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)] QGIS version: 3.6.2-Noosa Noosa, exported

Python Path: /usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python /Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python /Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins /usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/plugins /usr/local/opt/osgeo-gdal-python/lib/python3.7/site-packages /usr/local/opt/osgeo-qgis/QGIS.app/Contents/Resources/python /usr/local/opt/osgeo-qgis/lib/python3.7/site-packages /usr/local/opt/lib/python3.7/site-packages /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python37.zip /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7 /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/geos /usr/local/Cellar/osgeo-matplotlib/3.0.3_1/libexec/lib/python3.7/site-packages /usr/local/Cellar/numpy/1.16.3/libexec/nose/lib/python3.7/site-packages /usr/local/Cellar/protobuf/3.7.1/libexec/lib/python3.7/site-packages /Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python /usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions /usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions /usr/local/opt/osgeo-qgis/QGIS.app/Contents/MacOS/../Resources/python/IPython/extensions /Users/cedriclor/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/vector_tiles_reader/ext-libs`

mnboos commented 5 years ago

@sfkeller

What's the unique property of plugin's own version of protobuf which differentiates it from the already existant plugin? Just the fact that it's newer? What's the order on how newly registered libs are stored in the list?

Good questions. I'm not really proficient with protobuf. As far as I know, the protobuf version must match the version that was used to compile the protobufs.

@CedricLor Thanks for the report. At the moment I honestly don't know how to fix this.

dobrych commented 5 years ago

The problem seems to affect users that installed QGIS via brew, which has protobuf as dependency. So brew version conflicts with plugin-bundled version, and fix for me was: brew uninstall --ignore-dependencies protobuf protobuf-c

mnboos commented 4 years ago

Might be fixed with c56b6f8

Would anyone like to try it from the dev branch?