streamingfast / substreams-sink-kv

Substreams KV sink
Apache License 2.0
2 stars 5 forks source link

Can't run latest linux release #11

Closed YaroShkvorets closed 1 year ago

YaroShkvorets commented 1 year ago

https://github.com/streamingfast/substreams-sink-kv/releases/download/v0.1.2/substreams_sink_kv_linux_x86_64.tar.gz

It looks like it's dynamically linked and because of that I can't run it in docker

Getting this error: qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

Note: previsous version v0.1.1 worked fine

colindickson commented 1 year ago

Hello @YaroShkvorets .

I was not able to reproduce the issue in a pod running ubuntu 20.04:

colin@reader-1:~$ wget https://github.com/streamingfast/substreams-sink-kv/releases/download/v0.1.2/substreams_sink_kv_linux_x86_64.tar.gz
--2023-03-07 16:00:17--  https://github.com/streamingfast/substreams-sink-kv/releases/download/v0.1.2/substreams_sink_kv_linux_x86_64.tar.gz
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/585704265/9f1f5c9a-45a3-40e2-86a0-77e8e1d0d247?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230307%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230307T160018Z&X-Amz-Expires=300&X-Amz-Signature=d5fa0ae0ac06a52e5fcae710e51e2b3b7e16e2fb4fc7a2f5c7bd13aa2244a5a6&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=585704265&response-content-disposition=attachment%3B%20filename%3Dsubstreams_sink_kv_linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2023-03-07 16:00:18--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/585704265/9f1f5c9a-45a3-40e2-86a0-77e8e1d0d247?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230307%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230307T160018Z&X-Amz-Expires=300&X-Amz-Signature=d5fa0ae0ac06a52e5fcae710e51e2b3b7e16e2fb4fc7a2f5c7bd13aa2244a5a6&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=585704265&response-content-disposition=attachment%3B%20filename%3Dsubstreams_sink_kv_linux_x86_64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13763459 (13M) [application/octet-stream]
Saving to: ‘substreams_sink_kv_linux_x86_64.tar.gz’

substreams_sink_kv_linux_x86_64.tar.gz  100%[===============================================================================>]  13.12M  --.-KB/s    in 0.1s

2023-03-07 16:00:18 (122 MB/s) - ‘substreams_sink_kv_linux_x86_64.tar.gz’ saved [13763459/13763459]

colin@reader-1:~$ tar -xf substreams_sink_kv_linux_x86_64.tar.gz
colin@reader-1:~$ ls
LICENSE  README.md  substreams-sink-kv  substreams_sink_kv_linux_x86_64.tar.gz
colin@reader-1:~$ ./substreams-sink-kv
Substreams KV Sink

Usage:
  substreams-sink-kv [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  run         Fills a KV store from a substreams output and runs a Connect-Web listener

Flags:
      --delay-before-start duration   [OPERATOR] Amount of time to wait before starting any internal processes, can be used to perform to maintenance on the pod before actually letting it starts
  -h, --help                          help for substreams-sink-kv
      --metrics-listen-addr string    [OPERATOR] If non-empty, the process will listen on this address for Prometheus metrics request(s) (default "localhost:9102")
      --pprof-listen-addr string      [OPERATOR] If non-empty, the process will listen on this address for pprof analysis (see https://golang.org/pkg/net/http/pprof/) (default "localhost:6060")
  -v, --version                       version for substreams-sink-kv

Use "substreams-sink-kv [command] --help" for more information about a command.

What environment are you running docker on, and which ubuntu are you using as your base image?

YaroShkvorets commented 1 year ago

Try to docker compose up this docker-compose.yaml for example:

version: '3'
services:
  substreams-sink-test:
    image: ubuntu:latest
    container_name: "test-sink"
    command:
      - sh
      - -c
      - |
        apt-get update && apt-get install -y wget
        wget https://github.com/streamingfast/substreams-sink-kv/releases/download/v0.1.2/substreams_sink_kv_linux_x86_64.tar.gz -O - | tar -xz -C /usr/local/bin;
        substreams-sink-kv

Then replace the binary url with earlier version https://github.com/streamingfast/substreams-sink-kv/releases/download/v0.1.1/substreams-sink-kv_0.1.1_linux_x86_64.tar.gz - and it works

colindickson commented 1 year ago

ok thank you, i can now reproduce this issue. are you also running this docker compose on a mac?

YaroShkvorets commented 1 year ago

Yes, but I also had this issue when running it on Debian 11

colindickson commented 1 year ago

when you launch your docker compose up command, you should see the amd64 version of ubuntu being downloaded

test-sink  | Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
test-sink  | Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
test-sink  | Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [107 kB]
test-sink  | Get:5 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [5557 B]
test-sink  | Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [855 kB]
test-sink  | Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [17.5 MB]
test-sink  | Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [823 kB]
test-sink  | Get:9 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [883 kB]
test-sink  | Get:10 http://archive.ubuntu.com/ubuntu jammy/restricted amd64 Packages [164 kB]
test-sink  | Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages [1792 kB]
test-sink  | Get:12 http://archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [266 kB]
test-sink  | Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [880 kB]
test-sink  | Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [10.9 kB]
test-sink  | Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1189 kB]
test-sink  | Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1125 kB]
test-sink  | Get:17 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [49.0 kB]
test-sink  | Get:18 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [22.4 kB]

if not, set the following environment variable:

export DOCKER_DEFAULT_PLATFORM=linux/amd64

you might need to delete existing ubuntu docker images downloaded, and you might need to restart docker.

colindickson commented 1 year ago

marking this as closed. let me know if the issue persists.

YaroShkvorets commented 1 year ago

Couldn't get it to work with the flag. Anyway, not a big deal, running goreleaser produces a working linux binary I can use in docker, so I'll be using it instead of the one attached with v0.1.2 release. Thanks for looking into it.