search

turbot / steampipe-postgres-fdw

The Steampipe foreign data wrapper (FDW) is a zero-ETL product that provides Postgres foreign tables which translate queries into API calls to cloud services and APIs. It's bundled with Steampipe and also available as a set of standalone extensions for use in your own Postgres database.
https://steampipe.io/
Apache License 2.0
70 stars 16 forks source link

Lack of Docker image #168

Closed tnelson-doghouse closed 2 years ago

tnelson-doghouse commented 2 years ago

If there were a Docker image that contained this (building on top of, say, the official Postgres Docker image), that'd be great. To that end, here's what I have so far by way of a Dockerfile:

FROM postgres

ENV DEBIAN_FRONTEND=noninteractive
RUN apt update && apt install -y curl git make golang && rm -rf /var/lib/apt/lists/*

# Install Steampipe
RUN /bin/sh -c "$(curl -fsSL https://raw.githubusercontent.com/turbot/steampipe/main/install.sh)"

# Install the connector between Steampipe and Postgres
RUN cd /root && git clone https://github.com/turbot/steampipe-postgres-fdw.git
RUN cd /root/steampipe-postgres-fdw && ./build.sh

However, this fails as follows:

...
Step 6/7 : RUN cd /root/steampipe-postgres-fdw && ./build.sh
 ---> Running in 6b9f90ed744f
rm -f steampipe_postgres_fdw.so   libsteampipe_postgres_fdw.a  libsteampipe_postgres_fdw.pc
rm -f datum.o query.o fdw.o logging.o datum.bc query.bc fdw.bc logging.bc
rm -rf steampipe_postgres_fdw.a fdw.h
rm -rf results/ regression.diffs regression.out tmp_check/ tmp_check_iso/ log/ output_iso/
go build -o  steampipe_postgres_fdw.a -buildmode=c-archive ../*.go
go: downloading github.com/turbot/steampipe-plugin-sdk/v3 v3.0.1
go: downloading github.com/turbot/steampipe v0.13.0-rc.4
go: downloading github.com/golang/protobuf v1.5.2
go: downloading github.com/turbot/go-kit v0.3.0
go: downloading github.com/hashicorp/go-version v1.4.0
go: downloading google.golang.org/protobuf v1.27.1
go: downloading github.com/hashicorp/go-hclog v1.1.0
go: downloading google.golang.org/grpc v1.44.0
go: downloading github.com/btubbs/datetime v0.1.1
go: downloading github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading github.com/hashicorp/hcl/v2 v2.11.1
go: downloading golang.org/x/sys v0.0.0-20211210111614-af8b64212486
go: downloading github.com/dgraph-io/ristretto v0.1.0
go: downloading github.com/zclconf/go-cty v1.10.0
go: downloading github.com/fatih/color v1.13.0
go: downloading golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f
go: downloading github.com/fsnotify/fsnotify v1.5.1
go: downloading github.com/Masterminds/semver v1.5.0
go: downloading github.com/hashicorp/go-cleanhttp v0.5.2
go: downloading github.com/mattn/go-colorable v0.1.12
go: downloading github.com/Machiel/slugify v1.0.1
go: downloading github.com/olekukonko/tablewriter v0.0.5
go: downloading github.com/hashicorp/terraform v0.15.1
go: downloading github.com/sethvargo/go-retry v0.1.0
go: downloading github.com/spf13/viper v1.10.1
go: downloading golang.org/x/text v0.3.7
go: downloading github.com/mitchellh/mapstructure v1.4.3
go: downloading github.com/logrusorgru/aurora v2.0.3+incompatible
go: downloading github.com/gertd/go-pluralize v0.2.0
go: downloading github.com/mattn/go-runewidth v0.0.13
go: downloading github.com/jackc/pgx/v4 v4.15.0
go: downloading github.com/ghodss/yaml v1.0.0
go: downloading github.com/mattn/go-isatty v0.0.14
go: downloading sigs.k8s.io/yaml v1.3.0
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/containerd/containerd v1.4.12
go: downloading github.com/agext/levenshtein v1.2.2
go: downloading github.com/spf13/pflag v1.0.5
go: downloading gopkg.in/ini.v1 v1.66.2
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading github.com/apparentlymart/go-textseg v1.0.0
go: downloading github.com/jackc/pgconn v1.11.0
go: downloading github.com/stevenle/topsort v0.0.0-20130922064739-8130c1d7596b
go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0
go: downloading github.com/shiena/ansicolor v0.0.0-20200904210342-c7312218db18
go: downloading github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964
go: downloading github.com/google/go-cmp v0.5.6
go: downloading golang.org/x/crypto v0.0.0-20210921155107-089bfa567519
go: downloading github.com/jackc/chunkreader v1.0.0
go: downloading github.com/tkrajina/go-reflector v0.5.4
go: downloading github.com/google/uuid v1.3.0
go: downloading github.com/jackc/chunkreader/v2 v2.0.1
go: downloading github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b
go: downloading github.com/spf13/jwalterweatherman v1.1.0
go: downloading github.com/pelletier/go-toml v1.9.4
go: downloading github.com/deislabs/oras v0.8.1
go: downloading github.com/magiconair/properties v1.8.5
go: downloading github.com/zclconf/go-cty-yaml v1.0.2
go: downloading github.com/rivo/uniseg v0.2.0
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/briandowns/spinner v1.18.1
go: downloading github.com/opencontainers/image-spec v1.0.2
go: downloading github.com/hashicorp/go-plugin v1.4.3
go: downloading github.com/shirou/gopsutil v3.21.11+incompatible
go: downloading github.com/opencontainers/go-digest v1.0.0
go: downloading github.com/iancoleman/strcase v0.2.0
go: downloading github.com/jackc/pgio v1.0.0
go: downloading github.com/golang/glog v1.0.0
go: downloading github.com/dustin/go-humanize v1.0.0
go: downloading github.com/spf13/cast v1.4.1
go: downloading github.com/hashicorp/yamux v0.0.0-20181012175058-2f1d1f20f75d
go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
go: downloading github.com/cespare/xxhash v1.1.0
go: downloading github.com/jackc/pgproto3 v1.1.0
go: downloading github.com/tklauser/go-sysconf v0.3.9
go: downloading github.com/mitchellh/go-wordwrap v1.0.0
go: downloading github.com/cespare/xxhash/v2 v2.1.2
go: downloading github.com/jackc/pgproto3/v2 v2.2.0
go: downloading github.com/mitchellh/go-testing-interface v1.0.0
go: downloading github.com/karrick/gows v0.3.0
go: downloading github.com/subosito/gotenv v1.2.0
go: downloading google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa
go: downloading github.com/xlab/treeprint v1.1.0
go: downloading github.com/jackc/pgtype v1.10.0
go: downloading github.com/jackc/pgpassfile v1.0.0
go: downloading github.com/oklog/run v1.0.0
go: downloading github.com/spf13/afero v1.6.0
go: downloading github.com/tklauser/numcpus v0.3.0
go: downloading github.com/hashicorp/errwrap v1.1.0
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/hashicorp/go-uuid v1.0.1
go: downloading github.com/bmatcuk/doublestar v1.3.4
go: downloading github.com/apparentlymart/go-cidr v1.1.0
# github.com/turbot/steampipe/utils
/root/go/pkg/mod/github.com/turbot/steampipe@v0.13.0-rc.4/utils/fs_permissions.go:9:18: undefined: os.CreateTemp
/root/go/pkg/mod/github.com/turbot/steampipe@v0.13.0-rc.4/utils/wsl.go:16:27: undefined: os.ReadFile
note: module requires Go 1.17
make: *** [Makefile:30: go] Error 2
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fno-omit-frame-pointer -I./include/postgresql/server -I./include/postgresql/internal -g -fPIC -I. -I./ -I/usr/include/postgresql/14/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o datum.o datum.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fno-omit-frame-pointer -I./include/postgresql/server -I./include/postgresql/internal -g -fPIC -I. -I./ -I/usr/include/postgresql/14/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o query.o query.c
query.c:1:10: fatal error: steampipe_postgres_fdw.h: No such file or directory
    1 | #include "steampipe_postgres_fdw.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [<builtin>: query.o] Error 1
cp steampipe_postgres_fdw.so ~/.steampipe/db/12.1.0/postgres/lib/postgresql/steampipe_postgres_fdw.so
cp: cannot stat 'steampipe_postgres_fdw.so': No such file or directory
make: *** [Makefile:33: inst] Error 1
/root/steampipe-postgres-fdw
Removing intermediate container 6b9f90ed744f
 ---> ee893c60044c

I believe the key problem is currently fatal error: steampipe_postgres_fdw.h: No such file or directory. If anyone could assist with that, it'd be appreciated.

Thanks!

tnelson-doghouse commented 2 years ago

After some time looking at this, I'm coming to the following conclusions:

Again, any assistance appreciated.

Thanks!

tnelson-doghouse commented 2 years ago

Turns out I'm wrong; it just didn't show up in Google. See https://steampipe.io/docs/managing/containers for information.