turbot / steampipe-postgres-fdw

The Steampipe foreign data wrapper (FDW) is a zero-ETL product that provides Postgres foreign tables which translate queries into API calls to cloud services and APIs. It's bundled with Steampipe and also available as a set of standalone extensions for use in your own Postgres database.
https://steampipe.io/
Apache License 2.0
67 stars 16 forks source link

/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found while using cloudnativepg on top of fdw #495

Open sadathknorket opened 3 weeks ago

sadathknorket commented 3 weeks ago

Here is the error

ERROR:  could not load library "/usr/lib/postgresql/15/lib/steampipe_postgres_aws.so": /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /usr/lib/postgresql/15/lib/steampipe_postgres_aws.so)

I have the following Dockerfile for running postgres fdw

# Stage 1: Building the Go executable
FROM golang:1.22.2-bookworm AS build

WORKDIR /build

COPY ./ .

RUN go build -o main main.go

# Final image
FROM postgres:15

ENV TERM=xterm

RUN apt-get update && apt-get upgrade -y && \
    apt-get install -y curl && \
    rm -rf /var/lib/apt/lists/*

COPY install_fdw.sh /tmp/
RUN /tmp/install_fdw.sh && \
    rm /tmp/install_fdw.sh

COPY --from=build /build/main /
COPY --from=build /build/init.sh /docker-entrypoint-initdb.d

I decided to shift to cloudnativepg

and tried using postgres image with ghcr.io/cloudnative-pg/postgresql:15

Its very hard and error prone to upgrade GLIBC and I still did not succeed . Is there any other smart solution in place here?

sadathknorket commented 3 weeks ago

Here is my attempt to upgrade it

FROM golang:1.22.2-bookworm AS build

WORKDIR /build

COPY ./ .

RUN go build -o main main.go

# Stage 2: Install GLIBC 2.32 on CloudNativePG
FROM ghcr.io/cloudnative-pg/postgresql:15.4-3 AS final

ENV TERM=xterm
ENV GLIBC_VERSION=2.32

USER root

# Install dependencies for building GLIBC, including gawk and bison
RUN apt-get update && apt-get install -y \
    build-essential \
    wget \
    gcc \
    libc6-dev \
    libtool \
    make \
    gawk \
    bison \
    --no-install-recommends && \
    rm -rf /var/lib/apt/lists/*

# Download, compile, and install GLIBC 2.32
WORKDIR /tmp

RUN wget http://ftp.gnu.org/gnu/libc/glibc-${GLIBC_VERSION}.tar.gz && \
    tar -xzf glibc-${GLIBC_VERSION}.tar.gz && \
    cd glibc-${GLIBC_VERSION} && \
    mkdir build && cd build && \
    ../configure --prefix=/opt/glibc-${GLIBC_VERSION} && \
    make -j$(nproc) && \
    make install && \
    rm -rf /tmp/glibc-${GLIBC_VERSION}*

# Update the library path
ENV LD_LIBRARY_PATH=/opt/glibc-${GLIBC_VERSION}/lib:$LD_LIBRARY_PATH

# Copy the FDW installation script and run it
COPY install_fdw.sh /tmp/
RUN chmod +x /tmp/install_fdw.sh && \
    /bin/sh -c 'LD_LIBRARY_PATH=/opt/glibc-2.32/lib:$LD_LIBRARY_PATH /tmp/install_fdw.sh' && \
    rm /tmp/install_fdw.sh

# Copy the built Go binary and init script
COPY --from=build /build/main /
COPY --from=build /build/init.sh /docker-entrypoint-initdb.d

But does not work yet

pskrbasu commented 3 weeks ago

@sadathknorket Thanks for putting in the effort to write up this detailed issue.

From the details, it seems like you're trying to install steampipe_postgres_aws_15 extension. What is the glibc version used in your image?

I will try and replicate this.