search

C-X1 / meta-influx

Yocto Layer for influxdb etc. :warning: DEVELOPMENT :warning:
MIT License
0 stars 2 forks source link

Yocto 2.1.2 #1

Closed royemmerich closed 7 years ago

royemmerich commented 7 years ago

Hi @cyborg-x1

I'm currently trying to achieve the same goal, namely compiling influxdb under yocto 2.1.2 for arm. I'd really appreciate some feedback on how far you've got and whether you've achieved any level of success?

So far I've pretty much been following the instructions from here and my directory structure looks as follows:

../sources/oe-meta-go/
├── classes
│   └── go.bbclass
├── conf
│   ├── layer.conf
│   ├── multilib-example2.conf
│   └── multilib-example.conf
├── recipes-devtools
│   ├── examples
│   │   ├── files
│   │   │   └── helloworld.go
│   │   ├── go-examples.inc
│   │   └── go-helloworld_0.1.bb
│   └── go
│       ├── go
│       │   ├── 0001-cmd-ld-set-alignment-for-the-.rel.plt-section-on-32-.patch
│       │   ├── bsd_svid_source.patch
│       │   ├── ccache.patch
│       │   ├── do_not_create_native_cross_compilation_tools.patch
│       │   ├── Fix-ccache-compilation-issue.patch
│       │   └── fix-cross-compilation.patch
│       ├── go_1.4.3.inc
│       ├── go_1.6.3.bb
│       ├── go_1.6.3.inc
│       ├── go_1.7.4.bb -> go_1.6.3.bb
│       ├── go_1.7.4.inc
│       ├── go-bootstrap -> go
│       ├── go-bootstrap-native_1.4.3.bb
│       ├── go-cross -> go
│       ├── go-cross_1.4.3.bb
│       ├── go-cross_1.6.3.bb -> go-cross_1.7.4.bb
│       ├── go-cross_1.7.4.bb
│       ├── go-cross.inc
│       ├── go.inc
│       └── go-native_1.4.3.bb
├── recipes-extended
│   └── influxdb
│       ├── github.com-airbrake-gobrake.bb
│       ├── github.com-bmizerany-pat.bb
│       ├── github.com-boltdb-bolt.bb
│       ├── github.com-burntsushi-toml.bb
│       ├── github.com-cactus-go-statsd-client.bb
│       ├── github.com-cespare-xxhash.bb
│       ├── github.com-davecgh-go-spew.bb
│       ├── github.com-dgrijalva-jwt-go.bb
│       ├── github.com-dgryski-go-bits.bb
│       ├── github.com-dgryski-go-bitstream.bb
│       ├── github.com-gemnasium-logrus-airbrake-hook.bb
│       ├── github.com-gogo-protobuf.bb
│       ├── github.com-golang-snappy.bb
│       ├── github.com-influxdata-usage-client.bb
│       ├── github.com-jessevdk-go-flags.bb
│       ├── github.com-jwilder-encoding.bb
│       ├── github.com-paulbellamy-ratecounter.bb
│       ├── github.com-peterh-liner.bb
│       ├── github.com-rakyll-statik.bb
│       ├── github.com-retailnext-hllpp.bb
│       ├── github.com-sirupsen-logrus.bb
│       ├── github.com-stretchr-testify.bb
│       ├── github.com-uber-common-bark.bb
│       ├── github.com-uber-go-atomic.bb
│       ├── github.com-uber-go-zap.bb
│       ├── github.com-x-crypto.bb
│       └── influxdb_1.2.2.bb
└── recipes-net
    ├── cloud.google.com-go
    │   └── cloud.google.com-go.bb
    ├── github.com-golang-glog
    │   └── github.com-golang-glog.bb
    ├── github.com-golang-protobuf
    │   └── github.com-golang-protobuf.bb
    ├── golang.org-x-net
    │   └── golang.org-x-net.bb
    ├── golang.org-x-oauth2
    │   └── golang.org-x-oauth2.bb
    └── grpc
        └── grpc-go.bb

However the dependencies are driving me insane!

Any help is greatly appreciated.

royemmerich commented 7 years ago

In the meantime I have cloned this repo and am attempting to do a build.

github.com-googleapis-gax-go.bb claims to require google.golang.org/grpc:

src/github.com/googleapis/gax-go/call_option.go:36:2: cannot find package "google.golang.org/grpc" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/grpc (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0/src/google.golang.org/grpc (from $GOPATH)

I have added google.golang.org-grpc.bb:

DESCRIPTION = "google.golang.org/grpc"

GO_IMPORT = "google.golang.org/grpc"

inherit go

SRC_URI = "git://github.com/grpc/grpc-go;protocol=https;destsuffix=${PN}-${PV}/src/${GO_IMPORT}"
SRCREV = "6914ab1e338c92da4218a23d27fcd03d0ad78d46"
LICENSE = "BSD-3-Clause"
LIC_FILES_CHKSUM = "file://src/${GO_IMPORT}/LICENSE;md5=a4bad33881612090c6035d8393175996"

FILES_${PN} += "${GOBIN_FINAL}/*"

DEPENDS+="\
    github.com-gogo-protobuf \
"

But now I don't know how to solve the following error message:

ERROR: google.golang.org-grpc-1.0-r0 do_compile: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/temp/log.do_compile.3582)
ERROR: Logfile of failure stored in: /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/temp/log.do_compile.3582
Log data follows:
| DEBUG: Executing shell function do_compile
| GOARCH="arm"
| GOBIN=""
| GOEXE=""
| GOHOSTARCH="amd64"
| GOHOSTOS="linux"
| GOOS="linux"
| GOPATH="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0:/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go"
| GORACE=""
| GOROOT="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go"
| GOTOOLDIR="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/pkg/tool/linux_amd64"
| CC="arm-phytec-linux-gnueabi-gcc"
| GOGCCFLAGS="-fPIC -marm -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build217878831=/tmp/go-build -gno-record-gcc-switches"
| CXX="arm-phytec-linux-gnueabi-g++"
| CGO_ENABLED="0"
| src/google.golang.org/grpc/codec.go:40:2: cannot find package "github.com/golang/protobuf/proto" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/protobuf/proto (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/github.com/golang/protobuf/proto (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/protobuf/proto
| src/google.golang.org/grpc/status/status.go:49:2: cannot find package "google.golang.org/genproto/googleapis/rpc/status" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/genproto/googleapis/rpc/status (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/google.golang.org/genproto/googleapis/rpc/status (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/genproto/googleapis/rpc/status
| src/google.golang.org/grpc/credentials/oauth/oauth.go:42:2: cannot find package "golang.org/x/oauth2" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2 (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/golang.org/x/oauth2 (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2
| src/google.golang.org/grpc/credentials/oauth/oauth.go:43:2: cannot find package "golang.org/x/oauth2/google" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2/google (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/golang.org/x/oauth2/google (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2/google
| src/google.golang.org/grpc/credentials/oauth/oauth.go:44:2: cannot find package "golang.org/x/oauth2/jwt" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2/jwt (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/golang.org/x/oauth2/jwt (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/golang.org/x/oauth2/jwt
| src/google.golang.org/grpc/reflection/serverreflection.go:64:2: cannot find package "github.com/golang/protobuf/protoc-gen-go/descriptor" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/protobuf/protoc-gen-go/descriptor (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/github.com/golang/protobuf/protoc-gen-go/descriptor (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/protobuf/protoc-gen-go/descriptor
| src/google.golang.org/grpc/examples/helloworld/mock/mock_helloworld/hw_mock.go:7:2: cannot find package "github.com/golang/mock/gomock" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/mock/gomock (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/github.com/golang/mock/gomock (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/mock/gomock
| src/google.golang.org/grpc/grpclog/glogger/glogger.go:42:2: cannot find package "github.com/golang/glog" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/glog (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/google.golang.org-grpc-1.0/src/github.com/golang/glog (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/golang/glog
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/google.golang.org-grpc/1.0-r0/temp/log.do_compile.3582)
ERROR: Task 3921 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/google.golang.org-grpc.bb, do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1066 tasks of which 1065 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
No currently running tasks (1065 of 3928)

Summary: 1 task failed:
  /home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/google.golang.org-grpc.bb, do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.

How does one tell the google.golang.org-grpc.bb recipe where to look for the github.com/golang/protobuf/proto directory?

royemmerich commented 7 years ago

It would help if I was using the correct dependency:

DEPENDS+="\
    github.com-golang-protobuf \
"

I've made further progress but am stuck on another issue which I'll report about later if I can't solve it.

cyborg-x1 commented 7 years ago

Hi, It seems that influxdb is running on my raspberrypi... I haven't tested much so far cause I proceeded to telegraf. Except running influxdbd - which gave me the start screen with the logo ASCII - Art

Currently working only at my own docker recipe. The other stuff I should have. If you go to the deps of telegraf well - Godebs of telegraf you will get a little horror cause it has a lot more depends than influxdb.

I found out that in meta-virtualization there is a docker recipe but also there are other go recipes. Well - but they are not compatible to the naming of oe-meta-go. It will not recognize the already installed go compiler.

So you can not use it cause you will run into a conflict with the go version from oe-meta-go.

Also I found that I do not like the naming scheme from meta-virtualization: github.com-sirupsen-logrus.bb is there just go-logrus.bb

In this case not problematic but: if you do the recipes for influxdb... You find that you need:

github.com-gogo-protobuf.bb and github.com-golang-protobuf.bb

Here the story of the naming of meta-virtualization becomes a foot.

Two different things. Without the repo owner result in the same name.

I guess the go people always use the repo to identify the package. (I am not a go programmer - did not do a single line yet - just btw. )

Also the recipes are a lot more complicated.

I wrote actually a tool for creating recipes it is also in the repo here

it takes a file where each line is the repo and a secondary (now unused string - actually ment for the commit like in the depends file)

cyborg-x1 commented 7 years ago

To your question:

How does one tell the google.golang.org-grpc.bb recipe where to look for the github.com/golang/protobuf/proto directory?

You need a recipe for golang protobuf:

GO_IMPORT = "google.golang.org/grpc" specifies where it will go see my protobuf recipe

The proto dir comes from the repo itself.

cyborg-x1 commented 7 years ago

btw: the worst recipe I have so far.

royemmerich commented 7 years ago

Unfortunately I can't get your layer to compile out of the box on my system. I'm trying to figure out why but haven't yet been successful. Any tips greatly appreciated.

Here is the error I'm seeing:

ERROR: github.com-googleapis-gax-go-1.0-r0 do_compile: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.16416)
ERROR: Logfile of failure stored in: /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.16416
Log data follows:
| DEBUG: Executing shell function do_compile
| GOARCH="arm"
| GOBIN=""
| GOEXE=""
| GOHOSTARCH="amd64"
| GOHOSTOS="linux"
| GOOS="linux"
| GOPATH="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0:/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go"
| GORACE=""
| GOROOT="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go"
| GOTOOLDIR="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/pkg/tool/linux_amd64"
| CC="arm-phytec-linux-gnueabi-gcc"
| GOGCCFLAGS="-fPIC -marm -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build779208933=/tmp/go-build -gno-record-gcc-switches"
| CXX="arm-phytec-linux-gnueabi-g++"
| CGO_ENABLED="0"
| src/github.com/googleapis/gax-go/call_option.go:36:2: cannot find package "google.golang.org/grpc" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/grpc (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0/src/google.golang.org/grpc (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/grpc
| src/github.com/googleapis/gax-go/call_option.go:37:2: cannot find package "google.golang.org/grpc/codes" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/grpc/codes (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0/src/google.golang.org/grpc/codes (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/grpc/codes
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.16416)
ERROR: Task 3908 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1089 tasks of which 1088 didn't need to be rerun and 1 failed.
NOTE: Writing buildhistory
No currently running tasks (1088 of 3915)

Summary: 1 task failed:
  /home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
cyborg-x1 commented 7 years ago

ok thats strange I will have to check that when I am at home.

Probably it compiled for me without DEPENDS then O_o

You could try adding 

DEPENDS+="\
grpc-go \
"

to github.com-googleapis-gax-go

royemmerich commented 7 years ago

Based on the error above, this is what I attempted (naively).

I changed github.com-googleapis-gax-go.bb by adding the following to the bottom of the file:

DEPENDS +="\
    grpc-go \
"

Which resulted in the following error:

ERROR: Unbuildable tasks were found.
These are usually caused by circular dependencies and any circular dependency chains found will be printed below. Increase the debug level to see a list of unbuildable tasks.

Identifying dependency loops (this may take a short while)...

ERROR: 
Dependency loop #1 found:
  Task 2621 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_packagedata) (dependent Tasks ['grpc-go.bb, do_package'])
  Task 3913 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_package) (dependent Tasks ['github.com-googleapis-gax-go.bb, do_install', 'gcc-runtime, do_packagedata', 'pseudo, do_populate_sysroot', 'glibc, do_packagedata', 'rpm, do_populate_sysroot', 'grpc-go.bb, do_packagedata'])
  Task 3910 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_packagedata) (dependent Tasks ['github.com-googleapis-gax-go.bb, do_package'])
  Task 3846 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_package) (dependent Tasks ['gcc-runtime, do_packagedata', 'golang.org-x-net.bb, do_packagedata', 'github.com-googleapis-gax-go.bb, do_packagedata', 'pseudo, do_populate_sysroot', 'glibc, do_packagedata', 'rpm, do_populate_sysroot', 'cloud.google.com-go.bb, do_install'])
  Task 3843 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_packagedata) (dependent Tasks ['cloud.google.com-go.bb, do_package'])
  Task 3500 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_package) (dependent Tasks ['gcc-runtime, do_packagedata', 'golang.org-x-oauth2.bb, do_install', 'golang.org-x-net.bb, do_packagedata', 'pseudo, do_populate_sysroot', 'glibc, do_packagedata', 'cloud.google.com-go.bb, do_packagedata', 'rpm, do_populate_sysroot'])
  Task 3497 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_packagedata) (dependent Tasks ['golang.org-x-oauth2.bb, do_package'])
  Task 2624 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_package) (dependent Tasks ['glibc, do_packagedata', 'golang.org-x-net.bb, do_packagedata', 'gcc-runtime, do_packagedata', 'pseudo, do_populate_sysroot', 'golang.org-x-oauth2.bb, do_packagedata', 'rpm, do_populate_sysroot', 'github.com-golang-protobuf.bb, do_packagedata', 'grpc-go.bb, do_install', 'github.com-golang-glog.bb, do_packagedata'])

Dependency loop #2 found:
  Task 2615 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_install) (dependent Tasks ['pseudo, do_populate_sysroot', 'grpc-go.bb, do_compile'])
  Task 2616 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_populate_sysroot) (dependent Tasks ['grpc-go.bb, do_install'])
  Task 3907 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_configure) (dependent Tasks ['glibc, do_populate_sysroot', 'gcc-cross, do_populate_sysroot', 'grpc-go.bb, do_populate_sysroot', 'go-cross, do_populate_sysroot', 'gcc-runtime, do_populate_sysroot', 'github.com-googleapis-gax-go.bb, do_patch'])
  Task 3908 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_compile) (dependent Tasks ['github.com-googleapis-gax-go.bb, do_configure'])
  Task 3904 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_install) (dependent Tasks ['pseudo, do_populate_sysroot', 'github.com-googleapis-gax-go.bb, do_compile'])
  Task 3905 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_populate_sysroot) (dependent Tasks ['github.com-googleapis-gax-go.bb, do_install'])
  Task 3840 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_configure) (dependent Tasks ['golang.org-x-net.bb, do_populate_sysroot', 'github.com-googleapis-gax-go.bb, do_populate_sysroot', 'glibc, do_populate_sysroot', 'gcc-cross, do_populate_sysroot', 'go-cross, do_populate_sysroot', 'cloud.google.com-go.bb, do_patch', 'gcc-runtime, do_populate_sysroot'])
  Task 3841 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_compile) (dependent Tasks ['cloud.google.com-go.bb, do_configure'])
  Task 3837 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_install) (dependent Tasks ['pseudo, do_populate_sysroot', 'cloud.google.com-go.bb, do_compile'])
  Task 3838 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/cloud.google.com-go.bb, do_populate_sysroot) (dependent Tasks ['cloud.google.com-go.bb, do_install'])
  Task 3494 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_configure) (dependent Tasks ['golang.org-x-net.bb, do_populate_sysroot', 'golang.org-x-oauth2.bb, do_patch', 'glibc, do_populate_sysroot', 'gcc-cross, do_populate_sysroot', 'go-cross, do_populate_sysroot', 'gcc-runtime, do_populate_sysroot', 'cloud.google.com-go.bb, do_populate_sysroot'])
  Task 3495 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_compile) (dependent Tasks ['golang.org-x-oauth2.bb, do_configure'])
  Task 3491 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_install) (dependent Tasks ['pseudo, do_populate_sysroot', 'golang.org-x-oauth2.bb, do_compile'])
  Task 3492 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/golang.org-x-oauth2.bb, do_populate_sysroot) (dependent Tasks ['golang.org-x-oauth2.bb, do_install'])
  Task 2618 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_configure) (dependent Tasks ['golang.org-x-net.bb, do_populate_sysroot', 'github.com-golang-protobuf.bb, do_populate_sysroot', 'golang.org-x-oauth2.bb, do_populate_sysroot', 'glibc, do_populate_sysroot', 'gcc-cross, do_populate_sysroot', 'grpc-go.bb, do_patch', 'github.com-golang-glog.bb, do_populate_sysroot', 'gcc-runtime, do_populate_sysroot'])
  Task 2619 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb, do_compile) (dependent Tasks ['grpc-go.bb, do_configure'])

ERROR: Command execution failed: 1

Summary: There were 3 ERROR messages shown, returning a non-zero exit code.
cyborg-x1 commented 7 years ago

O_o What the f... why does this compile for me?

cyborg-x1 commented 7 years ago

Copied the dev recipes and put a file for go 1.8.1 well, lets say ... telegraf build ...

Just need to check if it runs and then cleanup... probably there are dependencies missing sometimes. I also stumpled over the one you mentioned ... don't know if it is fixed now...

royemmerich commented 7 years ago

Nope I still haven't solved it. I'm struggling to find where the circular dependency actually is.

royemmerich commented 7 years ago

By the way. In go-grpc.bb you have the following:

GO_INSTALL = "\
    ${GO_IMPORT}/grpc/codes/... \
    ${GO_IMPORT}/grpc/credentials/... \
    ${GO_IMPORT}/grpc/grpclog/... \
    ${GO_IMPORT}/grpc/internal/... \
    ${GO_IMPORT}/grpc/metadata/... \
    ${GO_IMPORT}/grpc/naming/... \
    ${GO_IMPORT}/grpc/peer/... \
    ${GO_IMPORT}/grpc/transport/... \
    ${GO_IMPORT}/genproto/googleapis/... \
    ${GO_IMPORT}/genproto/protobuf/... \
"

What do the trailing three dots (...) at the end of each line mean?

cyborg-x1 commented 7 years ago

Add all directories recursively... just a dash means without subdirs

It also complains when there are no buildable go files. So you have to exclude directories which do not build or contain no go files.

cyborg-x1 commented 7 years ago

Seems to run ... spectacle z22127

cyborg-x1 commented 7 years ago

Well no config so far I am afraid...

royemmerich commented 7 years ago

Thanks for your help so far @cyborg-x1 but I still can't see what is going on. I'm going to opt for the pre-compiled ARM binaries from the Influxdata website as I have other fish to fry right now.

cyborg-x1 commented 7 years ago

Sure ... I will clean up what I have now and push it tomorrow or so. I have fixed some recipes - also the one for where you had issues. But I need to add go 1.8.1 - actually I do not want to just copy the devtool section of oe-meta-go completely.

cyborg-x1 commented 7 years ago

if you want you can try one more: use this as oe-meta-go: https://github.com/cyborg-x1/oe-meta-go and the latest version of meta-influx

royemmerich commented 7 years ago

Okay I'll give it a try. Thanks very much for your help.

royemmerich commented 7 years ago

My word @cyborg-x1 , it worked!!! What did you change?

cyborg-x1 commented 7 years ago

Nice...

probably it was this one https://github.com/C-X1/meta-influx/commit/5a983bf2f236fedce7e404ca991034feb8e8d113

there where some dependencies missing in there, for myself I do not fully understand why it build for me in the first place. What ever... the rest was updating to go with 1.8.1

Currently the docker is not docker it is "just" the api for building telegraf

royemmerich commented 7 years ago

Great.

I have 2 further questions:

  1. I thought InflluxDB requires Go 1.7.4, not 1.8.1?

  2. InfluxDB has a lovely Godep file which tells you the exact state of its dependencies, but what about the sub-dependencies (and sub-sub-...) ? How do you know:

    • what their dependencies are, without building them and seeing the errors?
    • what commit hash to use in the Yocto recipe for each of these sub-dependencies? I'd think it rather crucial to know as it could mean the difference between a working and non-working InfluxDB binary?
cyborg-x1 commented 7 years ago

Well I guess you can use any of them, the thing is I actually wanted to update all of those packages to the latest revision. Unfortunately, when doing the recipes with my "automated" tool, I missed the fact that I still was using the revision given in the go file instead of the head which my tool gathered through checking out the repos. So most of the dependencies of telegraf are well at the state of the godep file.

I think normally there is only more functionality not less. That is also the reason why I had to use 1.8.1 because through me changing the tool to "head", it downloaded some to the latest revisions then...

And well of course those need 1.8.1 ... You know, I like bleeding edge tech ... (if it works :grin: )

The go guys seem to use the latest revision most of the times, or well it is just how I expect this go install command to work - where you just give it the repository ... and it does some "magic". But maybe I understood that wrong - do you have Go experience? :smile:

I actually have the idea to make another tool for python which updates all recipes of a directory to the latest revision (and maybe directly tries to compile them afterwards) ... Yeah I know, I am a bit crazy :stuck_out_tongue_winking_eye:

royemmerich commented 7 years ago

Unfortunately I don't have any Golang experience, but I'm learning FAST 😄.

About just using the latest commit of all dependencies: I'd strongly advise against doing that, regardless of the programming language. After all, InfluxDB didn't create a Godep file just for fun.

Cutting edge may be fine for testing out the latest functionality but when you're looking for reproducibility in a large scale production environment, just taking the head isn't the way to go. There must be a more reliable method than that.

royemmerich commented 7 years ago

@cyborg-x1 I realised I'd forgotten to include the influx layer in my local.conf file so I hadn't actually compiled it 😞 . I'm trying again now and unfortunately I have to report the following error:

ERROR: github.com-googleapis-gax-go-1.0-r0 do_compile: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.551)
ERROR: Logfile of failure stored in: /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.551
Log data follows:
| DEBUG: Executing shell function do_compile
| GOARCH="arm"
| GOBIN=""
| GOEXE=""
| GOHOSTARCH="amd64"
| GOHOSTOS="linux"
| GOOS="linux"
| GOPATH="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0:/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go"
| GORACE=""
| GOROOT="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go"
| GOTOOLDIR="/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/pkg/tool/linux_amd64"
| GCCGO="gccgo"
| GOARM=""
| CC="arm-phytec-linux-gnueabi-gcc"
| GOGCCFLAGS="-march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8 --sysroot=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1 -fPIC -marm -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build753805505=/tmp/go-build -gno-record-gcc-switches"
| CXX="arm-phytec-linux-gnueabi-g++"
| CGO_ENABLED="0"
| PKG_CONFIG="pkg-config"
| CGO_CFLAGS="-march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8 --sysroot=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1 -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0=/usr/src/debug/github.com-googleapis-gax-go/1.0-r0 -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux= -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1="
| CGO_CPPFLAGS=""
| CGO_CXXFLAGS="-march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8 --sysroot=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1 -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0=/usr/src/debug/github.com-googleapis-gax-go/1.0-r0 -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux= -fdebug-prefix-map=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1="
| CGO_FFLAGS="-g -O2"
| CGO_LDFLAGS="-march=armv7-a -marm -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a8 --sysroot=/home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1 -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
| src/google.golang.org/grpc/status/status.go:49:2: cannot find package "google.golang.org/genproto/googleapis/rpc/status" in any of:
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/x86_64-linux/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/genproto/googleapis/rpc/status (from $GOROOT)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/github.com-googleapis-gax-go-1.0/src/google.golang.org/genproto/googleapis/rpc/status (from $GOPATH)
|   /home/phyvm/yocto_builds_PD16.2.0/build/tmp/sysroots/phyboard-regor-am335x-1/usr/lib/arm-phytec-linux-gnueabi/go/src/google.golang.org/genproto/googleapis/rpc/status
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_compile (log file is located at /home/phyvm/yocto_builds_PD16.2.0/build/tmp/work/cortexa8hf-neon-phytec-linux-gnueabi/github.com-googleapis-gax-go/1.0-r0/temp/log.do_compile.551)
ERROR: Task 3909 (/home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/go/github.com-googleapis-gax-go.bb, do_compile) failed with exit code '1'
cyborg-x1 commented 7 years ago

Currently working on that .... well to be more precise on grpc-go, the root of this evil

cyborg-x1 commented 7 years ago

try the latest one... I guess it should be working now

royemmerich commented 7 years ago

Unfortunately still not luck:

ERROR: ParseError at /home/phyvm/yocto_builds_PD16.2.0/sources/poky/../meta-influx/recipes-go/grpc-go/grpc-go.bb:5: Could not inherit file classes/go.bbclass
cyborg-x1 commented 7 years ago

Never had this one, is the go layer not there? it says it does not find the go class of oe-meta-go. Have you tried cleaning the go stuff?

royemmerich commented 7 years ago

Indeed, I forgot to re-insert the oe-meta-go layer in bblayers.conf. Once I did that it all seems to have worked, with the following warnings:

WARNING: github.com-stretchr-testify-1.0-r0 do_populate_lic: github.com-stretchr-testify: No generic license file exists for: DWTFYW in any provider
WARNING: github.com-burntsushi-toml-1.0-r0 do_populate_lic: github.com-burntsushi-toml: No generic license file exists for: DWTFYW in any provider
WARNING: github.com-bmizerany-pat-1.0-r0 do_populate_lic: github.com-bmizerany-pat: No generic license file exists for: DWTFYW in any provider
WARNING: go-collectd.org-1.0-r0 do_populate_lic: go-collectd.org: No generic license file exists for: DWTFYW in any provider
WARNING: github.com-uber-go-zap-1.0-r0 do_package_qa: QA Issue: /usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/uber-go/zap/check_license.sh contained in package github.com-uber-go-zap-staticdev requires /bin/bash, but no providers found in RDEPENDS_github.com-uber-go-zap-staticdev? [file-rdeps]
WARNING: github.com-influxdata-influxdb-1.0-r0 do_package_qa: QA Issue: /usr/lib/arm-phytec-linux-gnueabi/go/src/github.com/influxdata/influxdb/build.py contained in package github.com-influxdata-influxdb-staticdev requires /usr/bin/python2.7, but no providers found in RDEPENDS_github.com-influxdata-influxdb-staticdev? [file-rdeps]

This is truly wonderful news! Thanks a ton for your help.

You mentioned something earlier about the InfluxDB config file. I presume I'll have to manually create one?

cyborg-x1 commented 7 years ago

At the moment yes. I have not tried if it works yet, btw. I also have a recipe for grafanas go stuff (backend) - but I am missing the front end stuff in nodeJS (did not have time yet)

cyborg-x1 commented 7 years ago

Well I tried launching it and it runs, but if there are issues, I don't know. If you put a config, tell me if it worked for you ;-)

royemmerich commented 7 years ago

Yip, it works for me too. However for production purposes we're rather going to use the pre-compiled influxdb binaries from the influxdata website.

cyborg-x1 commented 7 years ago

Okay, no problem