search

crowdsecurity / cs-firewall-bouncer

Crowdsec bouncer written in golang for firewalls
MIT License
119 stars 43 forks source link

Error building binary - go-re2 no Go source files #302

Closed JasonSanDiego closed 1 year ago

JasonSanDiego commented 1 year ago

I'm trying to build from source for Edgerouter-X following the instructions here

Here's what I get when attempting to build:

~/cs-firewall-bouncer$ make release
/bin/sh: 1: [: -gt: unexpected operator
/bin/sh: 3: [: -lt: unexpected operator
/bin/sh: 6: [: -lt: unexpected operator
go build -ldflags "-a -s -w -extldflags '-static' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.Version=v0.0.28-rc1' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.BuildDate=2023-06-24_14:52:15' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.Tag=c8322311d6486a54ff8791930d9735f8700bd296'" -trimpath -tags netgo  -o crowdsec-firewall-bouncer
build github.com/wasilibs/go-re2/internal/cre2: cannot load github.com/wasilibs/go-re2/internal/cre2: no Go source files
make: *** [Makefile:64: binary] Error 1
JasonSanDiego commented 1 year ago

I replicated this issue on another computer running Ubuntu 20.04 (WSL2).

Is there something wrong with the reference to the go-re2 dependency?

~/cs-firewall-bouncer$ make
/bin/sh: 1: [: -gt: unexpected operator
/bin/sh: 3: [: -lt: unexpected operator
/bin/sh: 6: [: -lt: unexpected operator
  go build -ldflags "-a -s -w -extldflags '-static' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.Version=v0.0.28-rc1' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.BuildDate=2023-06-28_22:57:34' -X 'github.com/crowdsecurity/go-cs-lib/pkg/version.Tag=c8322311d6486a54ff8791930d9735f8700bd296'" -trimpath -tags netgo  -o crowdsec-firewall-bouncer
  go: downloading github.com/sirupsen/logrus v1.9.2
  go: downloading golang.org/x/sync v0.2.0
  go: downloading golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc
  go: downloading github.com/crowdsecurity/go-cs-lib v0.0.2
  go: downloading gopkg.in/yaml.v2 v2.4.0
  go: downloading github.com/google/nftables v0.0.0-20220808154552-2eca00135732
  go: downloading github.com/crowdsecurity/crowdsec v1.5.2
  go: extracting golang.org/x/sync v0.2.0
  go: downloading golang.org/x/sys v0.8.0
  go: extracting github.com/sirupsen/logrus v1.9.2
  go: downloading github.com/crowdsecurity/go-cs-bouncer v0.0.7
  go: extracting gopkg.in/yaml.v2 v2.4.0
  go: extracting golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc
  go: downloading github.com/prometheus/client_golang v1.15.1
  go: extracting github.com/google/nftables v0.0.0-20220808154552-2eca00135732
  go: downloading github.com/mdlayher/netlink v1.6.0
  go: extracting github.com/crowdsecurity/go-cs-lib v0.0.2
  go: extracting github.com/crowdsecurity/go-cs-bouncer v0.0.7
  go: downloading gopkg.in/natefinch/lumberjack.v2 v2.2.1
  go: downloading github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f
  go: extracting golang.org/x/sys v0.8.0
  go: extracting github.com/prometheus/client_golang v1.15.1
  go: downloading github.com/prometheus/client_model v0.4.0
  go: downloading github.com/prometheus/common v0.44.0
  go: downloading github.com/beorn7/perks v1.0.1
  go: extracting github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f
  go: extracting gopkg.in/natefinch/lumberjack.v2 v2.2.1
  go: downloading github.com/cespare/xxhash v1.1.0
  go: downloading github.com/coreos/go-systemd/v22 v22.5.0
  go: extracting github.com/mdlayher/netlink v1.6.0
  go: extracting github.com/prometheus/client_model v0.4.0
  go: downloading google.golang.org/protobuf v1.30.0
  go: downloading github.com/josharian/native v1.0.0
  go: extracting github.com/crowdsecurity/crowdsec v1.5.2
  go: extracting github.com/beorn7/perks v1.0.1
  go: downloading github.com/mdlayher/socket v0.2.3
  go: extracting github.com/cespare/xxhash v1.1.0
  go: downloading github.com/cespare/xxhash/v2 v2.2.0
  go: extracting github.com/josharian/native v1.0.0
  go: downloading golang.org/x/net v0.10.0
  go: extracting github.com/prometheus/common v0.44.0
  go: extracting github.com/coreos/go-systemd/v22 v22.5.0
  go: downloading gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637
  go: downloading github.com/go-openapi/swag v0.22.3
  go: downloading github.com/go-openapi/errors v0.20.3
  go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4
  go: downloading github.com/pkg/errors v0.9.1
  go: extracting github.com/cespare/xxhash/v2 v2.2.0
  go: downloading github.com/goccy/go-yaml v1.11.0
  go: extracting google.golang.org/protobuf v1.30.0
  go: downloading github.com/go-openapi/strfmt v0.21.7
  go: downloading github.com/go-openapi/validate v0.22.1
  go: extracting gopkg.in/tomb.v2 v2.0.0-20161208151619-d5d1b5820637
  go: downloading github.com/prometheus/procfs v0.10.0
  go: extracting github.com/go-openapi/errors v0.20.3
  go: downloading github.com/google/go-querystring v1.1.0
  go: extracting github.com/pkg/errors v0.9.1
  go: downloading github.com/crowdsecurity/grokky v0.2.1
  go: extracting github.com/mdlayher/socket v0.2.3
  go: extracting github.com/go-openapi/swag v0.22.3
  go: downloading github.com/antonmedv/expr v1.12.5
  go: extracting github.com/matttproud/golang_protobuf_extensions v1.0.4
  go: downloading github.com/mailru/easyjson v0.7.7
  go: downloading gopkg.in/yaml.v3 v3.0.1
  go: extracting github.com/goccy/go-yaml v1.11.0
  go: extracting golang.org/x/net v0.10.0
  go: downloading github.com/golang/protobuf v1.5.3
  go: extracting github.com/go-openapi/strfmt v0.21.7
  go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
  go: extracting github.com/go-openapi/validate v0.22.1
  go: downloading github.com/mitchellh/mapstructure v1.5.0
  go: extracting github.com/google/go-querystring v1.1.0
  go: downloading go.mongodb.org/mongo-driver v1.11.6
  go: extracting github.com/prometheus/procfs v0.10.0
  go: downloading github.com/go-openapi/jsonpointer v0.19.6
  go: extracting github.com/mailru/easyjson v0.7.7
  go: extracting gopkg.in/yaml.v3 v3.0.1
  go: downloading github.com/go-openapi/spec v0.20.9
  go: downloading github.com/go-openapi/analysis v0.21.4
  go: downloading github.com/josharian/intern v1.0.0
  go: extracting github.com/crowdsecurity/grokky v0.2.1
  go: extracting github.com/golang/protobuf v1.5.3
  go: downloading github.com/go-openapi/loads v0.21.2
  go: extracting golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
  go: downloading github.com/oklog/ulid v1.3.1
  go: extracting github.com/go-openapi/jsonpointer v0.19.6
  go: extracting github.com/josharian/intern v1.0.0
  go: downloading github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
      go: extracting github.com/mitchellh/mapstructure v1.5.0
  go: downloading github.com/wasilibs/go-re2 v1.1.0
  go: extracting github.com/go-openapi/analysis v0.21.4
  go: downloading github.com/fatih/color v1.15.0
  go: extracting github.com/go-openapi/spec v0.20.9
  go: downloading github.com/go-openapi/jsonreference v0.20.2
  go: extracting github.com/go-openapi/loads v0.21.2
  go: extracting github.com/oklog/ulid v1.3.1
  go: extracting github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
       go: extracting github.com/fatih/color v1.15.0
  go: downloading github.com/mattn/go-isatty v0.0.19
  go: extracting github.com/go-openapi/jsonreference v0.20.2
  go: downloading github.com/mattn/go-colorable v0.1.13
  go: extracting github.com/mattn/go-isatty v0.0.19
  go: extracting github.com/mattn/go-colorable v0.1.13
  go: extracting github.com/wasilibs/go-re2 v1.1.0
  go: downloading github.com/tetratelabs/wazero v1.1.0
  go: extracting github.com/antonmedv/expr v1.12.5
  go: extracting go.mongodb.org/mongo-driver v1.11.6
  go: extracting github.com/tetratelabs/wazero v1.1.0
  go: finding github.com/sirupsen/logrus v1.9.2
  go: finding github.com/crowdsecurity/crowdsec v1.5.2
  go: finding golang.org/x/sys v0.8.0
  go: finding github.com/go-openapi/errors v0.20.3
  go: finding github.com/go-openapi/strfmt v0.21.7
  go: finding github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
  go: finding github.com/mitchellh/mapstructure v1.5.0
  go: finding github.com/oklog/ulid v1.3.1
  go: finding go.mongodb.org/mongo-driver v1.11.6
  go: finding github.com/go-openapi/swag v0.22.3
  go: finding github.com/mailru/easyjson v0.7.7
  go: finding github.com/josharian/intern v1.0.0
  go: finding gopkg.in/yaml.v3 v3.0.1
  go: finding github.com/go-openapi/validate v0.22.1
  go: finding github.com/go-openapi/analysis v0.21.4
  go: finding github.com/go-openapi/spec v0.20.9
  go: finding github.com/go-openapi/jsonpointer v0.19.6
  go: finding github.com/go-openapi/jsonreference v0.20.2
  go: finding github.com/go-openapi/loads v0.21.2
  go: finding github.com/crowdsecurity/go-cs-lib v0.0.2
  go: finding gopkg.in/yaml.v2 v2.4.0
  go: finding gopkg.in/natefinch/lumberjack.v2 v2.2.1
  go: finding github.com/prometheus/client_golang v1.15.1
  go: finding github.com/beorn7/perks v1.0.1
  go: finding github.com/cespare/xxhash/v2 v2.2.0
  go: finding github.com/prometheus/client_model v0.4.0
  go: finding google.golang.org/protobuf v1.30.0
  go: finding github.com/prometheus/common v0.44.0
  go: finding github.com/matttproud/golang_protobuf_extensions v1.0.4
  go: finding github.com/golang/protobuf v1.5.3
  go: finding github.com/prometheus/procfs v0.10.0
  go: finding github.com/google/nftables v0.0.0-20220808154552-2eca00135732
  go: finding github.com/mdlayher/netlink v1.6.0
  go: finding github.com/josharian/native v1.0.0
  go: finding github.com/mdlayher/socket v0.2.3
  go: finding golang.org/x/net v0.10.0
  go: finding golang.org/x/sync v0.2.0
  go: finding github.com/crowdsecurity/go-cs-bouncer v0.0.7
  go: finding github.com/goccy/go-yaml v1.11.0
  go: finding golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
  go: finding github.com/fatih/color v1.15.0
  go: finding github.com/mattn/go-colorable v0.1.13
  go: finding github.com/mattn/go-isatty v0.0.19
  go: finding github.com/antonmedv/expr v1.12.5
  go: finding github.com/crowdsecurity/grokky v0.2.1
  go: finding github.com/wasilibs/go-re2 v1.1.0
  go: finding github.com/tetratelabs/wazero v1.1.0
  build github.com/wasilibs/go-re2/internal/cre2: cannot load github.com/wasilibs/go-re2/internal/cre2: no Go source files
  make: *** [Makefile:64: binary] Error 1
mmetc commented 1 year ago

Hi @JasonSanDiego

the current master has a dependency on the re2 library, normally it should build the wasm version which does not require c++ or libre2-dev.

First, which version of go are you using? "go env GOVERSION" and "go version". There is a version detection which is failing at the top of your text.

To fix your issue you should run

$ go get github.com/crowdsecurity/crowdsec@ebe25d76 $ go mod tidy

This removes the dependency on re2, let me know if it worked

mmetc commented 1 year ago

as soon as we have a 1.5.3-rc1 tag on crowdsec, I'll update the dependency on master

JasonSanDiego commented 1 year ago

Thank you! I'm running go 1.13, which I take it is pretty old. Unfortunately, Ubuntu 20.04 doesn't have packages for a newer version of go, for whatever reason. I upgraded to 1.20 with a binary download and your instructions worked to build properly.