citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.17k stars 651 forks source link

ARM architecture support #3854

Open DaviesX opened 4 years ago

DaviesX commented 4 years ago

It seems like arm arch is not officially supported. Is it possible to build the extension from source on an arm platform?

techhazard commented 2 years ago

I just "sucesfully" built this on the 64-bit version of raspibian on a raspberry pi 4. make ran sucesfully, but then the regression tests had 5 failures. link to gist with regression.diff and regression.out

EDIT: some of the errors are probably locale related. I recompiled and ran the test again with LC_ALL=C and now only one test fails (multi_extension). I have updated the gist with the new logs.

techhazard commented 2 years ago

There is already a package available for citus 8 for postgres 11 on arm64:

pi@raspberry:~ $ apt info postgresql-11-citus
Package: postgresql-11-citus
Version: 8.0.0.PGDG-2
Priority: optional
Section: database
Source: citus
Maintainer: Citus Data <packaging@citusdata.com>
Installed-Size: 9850 kB
Depends: libc6 (>= 2.17), libcurl4 (>= 7.16.2), libpq5 (>= 9.2~beta3), postgresql-11
Homepage: https://github.com/citusdata/citus
Download-Size: 1230 kB
APT-Sources: http://deb.debian.org/debian buster/main arm64 Packages
Description: sharding and distributed joins for PostgreSQL
 Citus is a distributed database implemented as a PostgreSQL extension. It
 provides functions to easily split a PostgreSQL table into shards to be
 placed on remote worker nodes. Citus can replicate shards, update their
 schemas, and keep track of shard health. An advanced distributed planner
 is included which handles queries and modifications against sharded tables.
PeterFalken commented 1 year ago

I would be interested in getting docker images for linux/arm64v8 on dockerhuib similar to the linux/amd64 ones.

citusdata_docker

bartmika commented 1 year ago

Currently I am running a Apple M1 computer and this issue resolution would be a big help!

Since there is no arm64/v8 architecture, I am using amd64 and running into performance issues as the computer has to interpret the amd64 instruction set which results in slower Citus runtime.

Thank you so much for reading and Citus is awesome! Love it so far!

janantos commented 1 year ago

HI @bartmika I have altered docker file from https://github.com/zalando/patroni/tree/master/kubernetes so it builds and runs on M1 airbook

FROM postgres:15

RUN export DEBIAN_FRONTEND=noninteractive \
    && echo 'APT::Install-Recommends "0";\nAPT::Install-Suggests "0";' > /etc/apt/apt.conf.d/01norecommend \
    && apt-get update -y \
    && apt-get upgrade -y \
    && apt-cache depends patroni | sed -n -e 's/.* Depends: \(python3-.\+\)$/\1/p' \
            | grep -Ev '^python3-(sphinx|etcd|consul|kazoo|kubernetes)' \
            | xargs apt-get install -y busybox vim-tiny curl jq less locales git python3-pip python3-wheel postgresql-server-dev-15 autoconf flex git libcurl4-gnutls-dev libicu-dev \
                libkrb5-dev liblz4-dev libpam0g-dev libreadline-dev \
                libselinux1-dev libssl-dev libxslt1-dev libzstd-dev \
                 make uuid-dev build-essential
    ## Make sure we have a en_US.UTF-8 locale available
RUN localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 

RUN git clone https://github.com/citusdata/citus.git
WORKDIR citus
RUN ./configure
RUN make
RUN make install-all

RUN pip3 install setuptools \
    && pip3 install 'git+https://github.com/zalando/patroni.git#egg=patroni[kubernetes]' \
    && PGHOME=/home/postgres \
    && mkdir -p $PGHOME \
    && chown postgres $PGHOME \
    && sed -i "s|/var/lib/postgresql.*|$PGHOME:/bin/bash|" /etc/passwd \
    && /bin/busybox --install -s \
    # Set permissions for OpenShift
    && chmod 775 $PGHOME \
    && chmod 664 /etc/passwd \
    # Clean up
    && apt-get remove -y git python3-pip python3-wheel \
    && apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/* /root/.cache

ADD entrypoint.sh /
ENV PGSSLMODE=verify-ca PGSSLKEY=/etc/ssl/private/ssl-cert-snakeoil.key PGSSLCERT=/etc/ssl/certs/ssl-cert-snakeoil.pem PGSSLROOTCERT=/etc/ssl/certs/ssl-cert-snakeoil.pem

RUN sed -i 's/^postgresql:/&\n  basebackup:\n    checkpoint: fast/' /entrypoint.sh \
    && sed -i "s|^    postgresql:|&\n      pg_hba:\n      - local all all trust\n      - hostssl replication all all md5 clientcert=$PGSSLMODE\n      - hostssl all all all md5 clientcert=$PGSSLMODE\n      parameters:\n        max_connections: 100\n        shared_buffers: 16MB\n        ssl: 'on'\n        ssl_ca_file: $PGSSLROOTCERT\n        ssl_cert_file: $PGSSLCERT\n        ssl_key_file: $PGSSLKEY\n        citus.node_conninfo: 'sslrootcert=$PGSSLROOTCERT sslkey=$PGSSLKEY sslcert=$PGSSLCERT sslmode=$PGSSLMODE'|"  /entrypoint.sh \
    && sed -i "s#^    \(superuser\|replication\):#&\n      sslmode: $PGSSLMODE\n      sslkey: $PGSSLKEY\n      sslcert: $PGSSLCERT\n      sslrootcert: $PGSSLROOTCERT#" /entrypoint.sh

EXPOSE 5432 8008
ENV LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 EDITOR=/usr/bin/editor
USER postgres
WORKDIR /home/postgres
CMD ["/bin/bash", "/entrypoint.sh"]
a3kov commented 1 year ago

Has anybody tried running it on Ampere ? It's a very affordable option these days.

mustafaakin commented 4 months ago

Any updates or plans for brining ARM support to Citus? It looks like there used to be an ARM distribution but not anymore

rabiagulcan commented 1 month ago

@mustafaakin

Any updates or plans for brining ARM support to Citus? It looks like there used to be an ARM distribution but not anymore

merhaba, did you find any solution?