search

qgis / QGIS-Enhancement-Proposals

QEP's (QGIS Enhancement Proposals) are used in the process of creating and discussing new enhancements for QGIS
118 stars 37 forks source link

Support AWS Redshift cloud data warehouse in QGIS #272

Open reflectored opened 1 year ago

reflectored commented 1 year ago

QGIS Enhancement: Support AWS Redshift cloud data warehouse in QGIS

Date 2023/05/05

Author Alexey Karandashev (@reflectored)

Contact lexkar at amazon dot de

maintainer @reflectored

Version QGIS 3.X

Summary

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud that supports spatial types and functions defined by OGC Simple Feature Access Standard. AWS Redshift is available as a free trial.

Redshift enables the customer to analyze (spatial) data across warehouses, operational databases and data lakes, as well as creating and deploying machine learning models. Redshift customers that would like to visualize and work with the sheer amount of the data using QGIS currently require additional intermediate steps to visualize a limited amount of the data in QGIS.

We would like to provide the QGIS Redshift support to our current and future customers. QGIS exposure to Redshift customers will bring new users to QGIS.

Proposed Solution

We would like to add a new provider for AWS Redshift to QGIS. I have prepared a pull request for that.

I have looked into providing AWS Redshift support as a plugin. AWS Redshift driver requires a deeper integration, equivalent to other database providers in QGIS e.g. PostGIS, to achieve the same level of experience for our customers.

AWS is willing to provide support, AWS credits (for the CI Redshift provider tests) and everything that is necessary.

Affected Files

See the PR for a list of files.

Backwards Compatibility

I don't expect any backward compatibility issues.

Votes

nyalldawson commented 1 year ago

In the light of https://github.com/OSGeo/gdal/pull/8238, I would be very reluctant to see this merged 👎

@reflectored if you want open source software to support Amazon services, it might be a good idea to first push Amazon to support open source software...

haubourg commented 1 year ago

In the light of the previous experiences with proprietary and big vendors, where maintainers were forces to assume the maintenance and technical debt , I second Nyall's point of view. Please raise upstream in your organization that we need official and financial support to QGIS's project to have our maintainers able to review this work. Don't get me wrong, there is no anger. It is all about sustainability for us.

reflectored commented 1 year ago

@haubourg I understand your concern for ownership of the AWS Redshift provider.

AWS Redshift is officially committed to maintain the driver, I'm the current owner of the driver in the team.

Moreover, as me and @nyalldawson discussed previously here, I committed to refactoring the existing PostgreSQL driver while eliminating existing technical debt (#1 #2 #3). These and further contributions to the PostgreSQL driver manifest my commitment to the AWS Redshift driver.

In addition, the AWS Redshift provider build and tests, similarly to PostgreSQL, can be turned off by a simple flag in CMake. The reward of attracting many new users to QGIS, who personally voiced their demand for an AWS Redshift driver to me, outweighs the low risk of the driver's abandonment.

Regarding the financial support, I kindly ask you to elaborate on that subject in private, please send me an email to lexkar@amazon.de.

octave-ati commented 9 months ago

Any news on this? This is for sure something that my company would be interested in using