PeterPetrik commented 5 years ago

QGIS Enhancement: Static data providers

Date 2019/06/04

Author Peter Petrik (@PeterPetrik)

Contact peter.petrik@lutraconsulting.co.uk

maintainer @PeterPetrik

Version QGIS 3.10

Summary

Currently all QGIS data providers written in C++ are compiled as separate modules (.so / .dll) which are loaded by traversing lib/qgis/plugins directory upon start. The idea of this QEP is to make it possible to compile code of data providers directly into qgis_core library. This would become a compilation option which may be turned on or off.

Reasons:

simplification of bootstrap of qgis_core (no need to walk through plugins directory to load plugins one by one)
formalise internal API of c++ data providers
speeding up the load time for desktop builds (probably marginally)
iOS platform does not support dynamic libraries
for Android development, it would be more convenient to have data providers (OGR, GDAL, MDAL, postgresql, ...) as part of QGIS core (fewer .so files to ship)

This proposal describes technical solution of how to allow static linkage of providers to the core library.

Proposed Solution

The existing internal API for C++ data providers has become convoluted and consists of a bunch of exported functions that are not properly documented and data provider modules implement a subset of those. Moreover, some exported functions are quite specific to a particular provider.

Here is the list of exported functions

We propose to move OGR, GDAL and MDAL providers to src/core and force them to be always build similar to memory providers. Their extra functions used from qgisapp (e.g. directoryExtensions, isValidRasterFilename) will be outside the provider interface. QGIS cannot be built without OGR, GDAL and MDAL anyway and the number of functions that is exported only from these providers should not populate common provider's interface.

We propose to add the rest of the functions to QgsProviderMetadata similarly to CreateDataProviderFunction. We will deprecate use of createProviderLibrary and function() from QgsProviderRegistry, so it will not be possible to call custom functions, just functions from defined interface.

The interface can be divided to 5 function "groups":

core(description, type): core functions valid for any data provider
browser (dataItemProviders): functions related to browser panel
gui (selectWidget): functions related to GUI widgets
style (listStyles): functions for layer styles
data management(createTransaction): functions for database or data management handling

We introduce new CMAKE global define WITH_STATIC_PROVIDERS that will force all providers to be statically linked.

In dynamic-linked compilation, there will be just one C-exported function void* getQgsProviderMetadata() instead of current set of functions that will be checked by QgsProviderRegistry. In statically-linked compilation, the list of expected static providers will be hardcoded in the provider registry constructor.

Similarly to symbol layers (QgsSymbolLayerWidget), GUI components (widgets) will have metadata class. Registration of the GUI metadata will be done in qgis_gui initialisation routine.

Affected Files

many files all over the code that calls any of the custom OGR, GDAL and providers functions, most changes in

src/code/qgsproviderregistry.cpp
src/code/qgsprovidermetadata.cpp
src/providers
src/core/providers

Further Considerations/Improvements

remove dataItem() and dataCapabilities() functions
implement deleteStyleByID() for msqsl, spatialite and oracle providers
merge classFactory() and createProvider() functions

Backwards Compatibility

deprecate use of createProviderLibrary and function() from QgsProviderRegistry
custom/external C++ providers will need to implement new interface

Issue Tracking ID(s)

(optional)

Votes

(required)

elpaso commented 5 years ago

big +1!

luipir commented 5 years ago

generally +0, but +1 due to "iOS platform does not support dynamic libraries"

nyalldawson commented 5 years ago

+1 to moving gdal/ogr providers to core. It's impossible to build QGIS without gdal anyway, and I can't see any good reason why anyone would NOT want to use GDAL/OGR layers, so I can't see any reason for keeping these out of core.

NathanW2 commented 5 years ago

+1

On Tue, Jun 4, 2019 at 8:05 PM Nyall Dawson notifications@github.com wrote:

+1 to moving gdal/ogr providers to core. It's impossible to build QGIS without gdal anyway, and I can't see any good reason why anyone would NOT want to use GDAL/OGR layers, so I can't see any reason for keeping these out of core.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/149?email_source=notifications&email_token=AAC5FXEUAK5TTIEWABLFLPTPYY46ZA5CNFSM4HSZIWFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW4CO2Q#issuecomment-498607978, or mute the thread https://github.com/notifications/unsubscribe-auth/AAC5FXHHG4TU5BDFWE53XHTPYY46ZANCNFSM4HSZIWFA .