Logs are being sent to `stderr`

[gusinacio]

I'm running substreams-sink-postgres on GCP via a Kubernetes and a Deployment with 1 Pod. All the logs are being recognized as error, which means that all the logs are being sent to stderr.

You can reproduce by saving the stderr into a file:

substreams-sink-postgres run <database> <host> <spkg> db_out 2> stderr.log

[maoueh]

That is the expected behavior, logging is often performed on stderr. We do run also on Kubernetes and I don't remember seeing this.

Do you see the log formatted as JSON at least on your deployment?

[gusinacio]

[gusinacio]

All of them have "severity: "ERROR" and no JSON, the record is textPayload.

[maoueh]

Yeah so the problem is that it still prints as text. Our logging library normally would have detected that your are in a container environment and switched to JSON which I think would solve the issue for you.

Are you building from master?

[gusinacio]

This is the dockerfile:

FROM rust:1 as build
ARG TARGET

ENV SUBSTREAMS_VERSION=v1.1.5

# install protobuf

RUN apt-get update && \
            apt-get install -y protobuf-compiler &&\
            rm -rf /var/lib/apt/lists/*
ENV BUF_VERSION="1.21.0"
RUN curl -sSL \
            "https://github.com/bufbuild/buf/releases/download/v${BUF_VERSION}/buf-$(uname -s)-$(uname -m)" \
            -o "/usr/local/bin/buf" && \
            chmod +x "/usr/local/bin/buf"
# install wasm32 toolchain
RUN rustup target add wasm32-unknown-unknown
RUN wget https://github.com/streamingfast/substreams/releases/download/$SUBSTREAMS_VERSION/substreams_linux_x86_64.tar.gz && \
                tar -xvf substreams_linux_x86_64.tar.gz && \
                mv substreams /usr/local/bin
WORKDIR /app

# dependencies installation
RUN mkdir src/ && touch ./src/lib.rs
COPY ./Cargo.lock ./Cargo.toml ./
RUN cargo build --release && rm src/*.rs

# code compilation
COPY ./Makefile ./substreams.yaml ./
COPY ./proto ./proto
COPY ./src ./src

RUN make compile

FROM debian:bullseye as application
ARG TARGET

ENV SUBSTREAMS_POSTGRES_VERSION=2.3.2
ENV SUBSTREAMS_GIT_REPO=https://github.com/streamingfast/substreams-sink-postgres/releases/download

RUN apt-get update && \
            apt-get install -y wget \
                               make && \
            rm -rf /var/lib/apt/lists/*

# install substreams-sink-postgres
RUN wget $SUBSTREAMS_GIT_REPO/$SUBSTREAMS_POSTGRES_VERSION/substreams-sink-postgres_linux_x86_64.tar.gz && \
            tar -xvf substreams-sink-postgres_linux_x86_64.tar.gz && \
            mv substreams-sink-postgres /usr/local/bin

WORKDIR /app

COPY --from=build /app/substreams.spkg ./substreams.spkg
COPY ./schema.sql ./Makefile ./

# setup + sink
ENTRYPOINT ["make", "execute"]

The latest version that I tested directly from releases was 2.3.2. Since then I was using the branches that I was working on. Usually they were synced with the master branch from streamingfast. I never got a jsonPayload in GCP.

[maoueh]

Can you retry to build from develop branch, I've just bumped to use our latest logging library which once you re-built your container with it, it should be good now.

[gusinacio]

Could you create a patch release or a prerelease? My current dockerfile is tightly attached to your releases

[maoueh]

I just released 2.5.2, let me know how it looks like.

[gusinacio]

Thank you! Everything is working smoothly.