search

hyperledger-labs / blockchain-analyzer

Analyze ledger data stored within a Hyperledger Fabric peer (key updates and operational data such as number of blocks and transactions).
Apache License 2.0
23 stars 17 forks source link

fabricbeat 'make update' #17

Closed lotty02cho closed 4 years ago

lotty02cho commented 5 years ago

I try to start make update but this error comes out.

/opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat$ make update
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Installing mage v1.8.0 from vendor dir.
can't load package: package /opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat/vendor/github.com/magefile/mage: import "/opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat/vendor/github.com/magefile/mage": cannot import absolute path
vendor/github.com/elastic/beats/dev-tools/make/mage.mk:9: recipe for target 'mage' failed
make: *** [mage] Error 1
salmanbaset commented 5 years ago

@lotty02cho after the move to hyperledger-labs, the agent paths need to be updated from hyperledger-elastic to blockchain-analyzer. They are being updated in this PR, which will be merged tomorrow. cc @balazsprehoda

salmanbaset commented 5 years ago

@lotty02cho please full the latest changes from master. Hopefully, the make update should work.

Please note that go module vendoring is still a work in progress. Please copy vendor modules from this previous commit to the agent vendor directory: https://github.com/hyperledger-labs/blockchain-analyzer/tree/14199bfa3e770822a7877652d1a589b425b94894/agent/fabricbeat/vendor/github.com

and then run make .

lotty02cho commented 5 years ago

I try to clone 'blockchain-analyzer' project, but it has still errors.

/opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat$ make update
Running virtualenv with interpreter /usr/bin/python2
New python executable in /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/build/python-env/bin/python2
Also creating executable in /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/build/python-env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: docker-compose 1.23.2 has requirement PyYAML<4,>=3.10, but you'll have pyyaml 4.2b1 which is incompatible.
Installing mage v1.8.0 from vendor dir.
can't load package: package /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/magefile/mage: import "/opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/magefile/mage": cannot import absolute path
vendor/github.com/elastic/beats/dev-tools/make/mage.mk:9: recipe for target 'mage' failed
make: *** [mage] Error 1

I think that absolute path is important, do I have to change path or absolute path like that?

balazsprehoda commented 4 years ago

@lotty02cho I could not reproduce this error, but I suspect that the Makefile in the fabricbeat dir is responsible for this. Try to change the BEAT_PATH assginment to BEAT_PATH=github.com/blockchain-analyzer/agent/fabricbeat and see if the error is gone. Please let me know if this helps, and I fix it in a PR.

lotty02cho commented 4 years ago

As you said, I change BEAT_PATH=github.com/blockchain-analyzer/agent/fabricbeat using vi Makefile. and now I couldn't find error.

/opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat$ make update
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Installing mage v1.8.0 from vendor dir.
/home/hyper/.magefile cleaned
Generated fields.yml for fabricbeat to /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/fields.yml
Updating generated files for fabricbeat
## The line below allows beats to add a post-processor script that alters the config in arbitrary ways
mkdir -p include
go run  ./vendor/github.com/elastic/beats/dev-tools/cmd/asset/asset.go -license ASL2 -pkg include -in fields.yml -out include/fields.go fabricbeat
if [ -d /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/_meta/kibana ]; then \
    cp -pr /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/_meta/kibana/* /opt/gopath/src/github.com/blockchain-analyzer/agent/fabricbeat/_meta/kibana.generated ; \
fi

but when I start next command, make, it throws error like below.

...
../../../hyperledger/fabric-sdk-go/pkg/fab/comm/connectionopts.go:16:2: cannot find package "google.golang.org/grpc/keepalive" in any of:
    /opt/go/src/google.golang.org/grpc/keepalive (from $GOROOT)
    /opt/gopath/src/google.golang.org/grpc/keepalive (from $GOPATH)
../../../hyperledger/fabric-sdk-go/pkg/fab/comm/streamconnection.go:20:2: cannot find package "google.golang.org/grpc/peer" in any of:
    /opt/go/src/google.golang.org/grpc/peer (from $GOROOT)
    /opt/gopath/src/google.golang.org/grpc/peer (from $GOPATH)
../../../hyperledger/fabric-sdk-go/pkg/common/errors/status/status.go:22:2: cannot find package "google.golang.org/grpc/status" in any of:
    /opt/go/src/google.golang.org/grpc/status (from $GOROOT)
    /opt/gopath/src/google.golang.org/grpc/status (from $GOPATH)
../../../hyperledger/fabric-sdk-go/internal/github.com/hyperledger/fabric/msp/configbuilder.go:23:2: cannot find package "gopkg.in/yaml.v2" in any of:
    /opt/go/src/gopkg.in/yaml.v2 (from $GOROOT)
    /opt/gopath/src/gopkg.in/yaml.v2 (from $GOPATH)
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:107: recipe for target 'fabricbeat' failed
make: *** [fabricbeat] Error 1

when I access fabric-sdk-go directory, path is correct. Is there any path or value that I have to change in Makefile?

salmanbaset commented 4 years ago

@lotty02cho see instructions here on building the agent on Ubuntu 16.04/18.04 or Mac OS X: https://github.com/hyperledger-labs/blockchain-analyzer/blob/master/docs/Basic_setup.md#build-fabricbeat-agent

ashishxooa commented 4 years ago

I see an error in the documentation in building the agent. It should be "make go-get" and not "make get-go"

but that also is throwing errors.

➜ fabricbeat git:(master) ✗ make go-get go get github.com/go-kit/kit/sd go: finding github.com/cloudflare/cfssl 1.3.3 go: finding github.com/go-kit/kit vv0.8.0 go: errors parsing go.mod: /src/github.com/blockchain-analyzer/agent/fabricbeat/go.mod:43: invalid module version "vv0.8.0": unknown revision vv0.8.0 make: *** [go-get] Error 1

balazsprehoda commented 4 years ago

@ashishxooa Thanks for the feedback. The error you get after using make go-get is probably caused by this line. Instead of vv0.8.0, it should be v0.8.0. Please try that, and let us know if it helps.

ashishxooa commented 4 years ago

@balazsprehoda Hi yes, I saw that bug yesterday and corrected that. Even after doing that change issues.

go: finding github.com/go-kit/kit/sd latest cd ../../../go-kit/kit/; git checkout tags/v0.8.0 -b v0.8.0 bash: line 0: cd: ../../../go-kit/kit/: No such file or directory fatal: 'tags/v0.8.0' is not a commit and a branch 'v0.8.0' cannot be created from it make: *** [go-get] Error 128

SwapnilEI commented 4 years ago

@balazsprehoda Hi yes, I saw that bug yesterday and corrected that. Even after doing that change issues.

go: finding github.com/go-kit/kit/sd latest cd ../../../go-kit/kit/; git checkout tags/v0.8.0 -b v0.8.0 bash: line 0: cd: ../../../go-kit/kit/: No such file or directory fatal: 'tags/v0.8.0' is not a commit and a branch 'v0.8.0' cannot be created from it make: *** [go-get] Error 128

@ashishxooa @balazsprehoda @salmanbaset I'm also facing the same issue, not able to proceed further to get fabricbeat-* in Kibana like said in Readme doc. :(

balazsprehoda commented 4 years ago

@ashishxooa @SwapnilEI @salmanbaset I tried to reproduce this error in a new Ubuntu 18.04 VM, but I found a few other issues instead :) I have submitted a PR to fix those. For this error, I would recommend to check if you can manually clone go-kit and check out v0.8.0. Sorry for the late answer.

swapnil-kogx commented 4 years ago

@ashishxooa @salmanbaset @balazsprehoda Thanks for the reply and fixing that issue. But, now it gave new error as it could not find protoutil. :( Here is the error:

go get github.com/hyperledger/fabric/protoutil: module github.com/hyperledger/fabric@upgrade found (v1.4.4), but does not contain package github.com/hyperledger/fabric/protoutil make: *** [go-get] Error 1

Also, @ashishxooa were you able to proceed further?

balazsprehoda commented 4 years ago

@lotty02cho @ashishxooa @SwapnilEI @salmanbaset I uploaded an executable built on Ubuntu 18.04 to my fork. I am going to upload another for Ubuntu 16.04 soon. After that, I am going to submit a PR. Until then, please take a look at this and see if it helps.

ashish235 commented 4 years ago

@swapnil-kogx I did delete everything and started everything from scratch and this time it worked. I use a mac so it got built on mac. trying now on Ubuntu 18.04 maybe @balazsprehoda fork will help.

balazsprehoda commented 4 years ago

@ashish235 @swapnil-kogx @ashishxooa We have released a Docker image of fabricbeat which makes deployment easier (you do not have to build it on your own machine). For details, see #37