search

vertica / spark-connector

This component acts as a bridge between Spark and Vertica, allowing the user to either retrieve data from Vertica for processing in Spark, or store processed data from Spark into Vertica.
Apache License 2.0
20 stars 23 forks source link

Support Vertica on ARM #506

Open jeremyprime opened 1 year ago

jeremyprime commented 1 year ago

Is your feature request related to a problem? Please describe.

Vertica is currently moving towards supporting ARM processors. We need to support Vertica and other development containers (Spark, Hadoop, etc) on ARM as well to avoid performance issues.

Describe the solution you'd like

Once the version of Vertica that supports ARM is released, create an ARM-based Docker images for Vertica, Spark, Hadoop, etc.

We also want to run all of our tests to ensure the Spark Connector does not have any issues when run on ARM against a Vertica (also on ARM).

Additional context

Note that we may need to go to our current images, such as Bitnami Spark, and build ARM versions ourselves since most images do not currently provide ARM versions.

NerdLogic commented 1 year ago

Wait until VERTICA, Bitnami, and hdfs are available on arm. Bitnami images look like they might have been released recently.

alexey-temnikov commented 1 year ago

Once issue is priorities - need to consider ticket - https://github.com/vertica/spark-connector/issues/472