search

icon-project / Nexus

7 stars 3 forks source link

Setup BSC with current testnet #308

Closed tiendq closed 3 years ago

tiendq commented 3 years ago

Setup BSC to work with current ICON/Moonbeam testnet.

mohamedelshami commented 3 years ago

Hi @tiendq,

Have you tried running the docker following steps in bsc guide.

tiendq commented 3 years ago

@mohamedelshami I looked at it one time as mentioned in #273 and there were several things unclear at we posted. We haven't tried again since there is no feedback from your team (on #273). BTW, @sanglt1902 will handle this task.

tiendq commented 3 years ago

@mohamedelshami @simsonraj

I noticed your guide is 7 day old, no more updates/fixes last week? I'd like the latest/most stable one to give it a try. Thanks.

tiendq commented 3 years ago

@mohamedelshami @simsonraj

I have several questions after review what @sanglt1902 tried last week, please reply each separately so make sure we're clear about them.

  1. No goloop branch v0.9.7 (command doesn't work with tag).
  2. Are they different, OpenJDK 11 vs. JDK11? Our current testnet requires JDK11
  3. Is NodeJS 14.x LTS fine? Requiring the latest NodeJS version might affect our web apps and other JS tools.
  4. I see this guide creates its own ICON network, can it be configured to integrate to our existing local ICON network?
  5. It needs to be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-915049744)
  6. It needs to be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-915100350)
  7. These verifications should be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-916697472)
  8. Is it fixed? (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-916713424)

And the last issue reported here? How is about it now?

simsonraj commented 3 years ago

  1. No goloop branch v0.9.7 (command doesn't work with tag). Do you face issues with clone command "git -b"? It works fine for us.

  2. Are they different, OpenJDK 11 vs. JDK11? Our current testnet requires JDK11 OpenJDK11 is ICON's recommendation for the Javascore compilation, if your existing setup works with Icondao's javascore, this shouldn't be a problem.

  3. Is NodeJS 14.x LTS fine? Requiring the latest NodeJS version might affect our web apps and other JS tools. Node.js V14.x works just fine.

  4. I see this guide creates its own ICON network, can it be configured to integrate to our existing local ICON network? Assuming, Goloop container build is the question about, the provision scripts can be modified to point to existing goloop network.

  1. Change the  GOLOOP_RPC_URI & GOLOOP_RPC_ADMIN_URI env variables to point to new URI in provision.sh
  2. Change the BTPSIMPLE_SRC_ENDPOINT in the docker-compose.yml file & comment out the goloop service in the yaml definition.
  3. Place the goloop.keystore.json & goloop.keysecret files under "devnet/docker/icon-bmc/work" folder
  1. It needs to be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-915049744) - Done
  2. It needs to be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-915100350) - Done
  3. These verifications should be included in the guide (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-916697472)
  4. Is it fixed? (https://github.com/icon-project/btp-dashboard/issues/273#issuecomment-916713424) For 7 & 8, This is not an issue with the provision script, this doesn’t happen with our tests, Can we suggest trying it again from scratch and if possible on a different machine.
tiendq commented 3 years ago

@simsonraj

  1. No, it didn't work but it was just syntax, I could checkout v0.9.7 tag.
  2. So I assume JDK11 is fine
  3. Great to hear your confirmation "Node.js V14.x works just fine"
  4. I think you should provide this ability since our testnet needs to have all networks in one deployment. But it's fine for now, at least until I can make BSC works

"11". Yes, actually I planned to try it from scratch on a new machine

Other things: fine.

Thanks for your answers.

tiendq commented 3 years ago

@simsonraj

Image is created but I got following error (in red) with this command docker build --tag bsc-node ./devnet/docker/bsc-node --build-arg KEYSTORE_PASS=Perlia0

Error:

Step 11/15 : RUN ./provision.sh
 ---> Running in 5159698cd65d
INFO [09-16|04:02:02.981] Maximum peer count                       ETH=50 LES=0 total=50
INFO [09-16|04:02:02.981] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=ledger vendor=11415 failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=21324 failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=4617  failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=ledger vendor=11415 failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=21324 failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.983] Failed to enumerate USB devices          hub=trezor vendor=4617  failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
INFO [09-16|04:02:02.983] Allocated cache and file handles         database=/bsc/node1/geth/chaindata cache=16.00MiB handles=16
INFO [09-16|04:02:02.988] Writing custom genesis block 
INFO [09-16|04:02:02.990] Persisted trie from memory database      nodes=27 size=81.47KiB time=806.09µs gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-820.00B
INFO [09-16|04:02:02.990] Successfully wrote genesis state         database=chaindata                 hash=a0b4bf…36da90
INFO [09-16|04:02:02.990] Allocated cache and file handles         database=/bsc/node1/geth/lightchaindata cache=16.00MiB handles=16
INFO [09-16|04:02:02.996] Writing custom genesis block 
INFO [09-16|04:02:02.997] Persisted trie from memory database      nodes=27 size=81.47KiB time=271.457µs gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-820.00B
INFO [09-16|04:02:02.997] Successfully wrote genesis state         database=lightchaindata                 hash=a0b4bf…36da90
Removing intermediate container 5159698cd65d
 ---> 115ccff85966
Step 12/15 : COPY ./entrypoint ${BSC_HOME}
tiendq commented 3 years ago

@simsonraj

I did go mod download github.com/ethereum/go-ethereum but still got many errors like these with make under ./btp at this step:

Makefile:133: warning: overriding recipe for target 'test'
Makefile:50: warning: ignoring old recipe for target 'test'
[#] go build ./cmd/btpsimple
CGO_ENABLED=0 go build -tags "" -ldflags "-X 'main.version=1503db2-dirty' -X 'main.build=linux/amd64 tags()-2021-09-16-04:11:57'" -o ./bin/btpsimple ./cmd/btpsimple
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/core/state/database.go:23:2: missing go.sum entry for module providing package github.com/VictoriaMetrics/fastcache (imported by github.com/ethereum/go-ethereum/core/state); to add:
    go get github.com/ethereum/go-ethereum/core/state@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/crypto/signature_nocgo.go:28:2: missing go.sum entry for module providing package github.com/btcsuite/btcd/btcec (imported by github.com/ethereum/go-ethereum/crypto); to add:
    go get github.com/ethereum/go-ethereum/crypto@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/rpc/server.go:24:2: missing go.sum entry for module providing package github.com/deckarep/golang-set (imported by github.com/ethereum/go-ethereum/accounts/keystore); to add:
    go get github.com/ethereum/go-ethereum/accounts/keystore@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/consensus/ethash/ethash.go:36:2: missing go.sum entry for module providing package github.com/edsrzf/mmap-go (imported by github.com/ethereum/go-ethereum/consensus/ethash); to add:
    go get github.com/ethereum/go-ethereum/consensus/ethash@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/accounts/scwallet/hub.go:48:2: missing go.sum entry for module providing package github.com/gballet/go-libpcsclite (imported by github.com/ethereum/go-ethereum/accounts/scwallet); to add:
    go get github.com/ethereum/go-ethereum/accounts/scwallet@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/log/handler.go:11:2: missing go.sum entry for module providing package github.com/go-stack/stack (imported by github.com/ethereum/go-ethereum/log); to add:
    go get github.com/ethereum/go-ethereum/log@v1.10.2
simsonraj commented 3 years ago

@simsonraj

Image is created but I got following error (in red) with this command docker build --tag bsc-node ./devnet/docker/bsc-node --build-arg KEYSTORE_PASS=Perlia0

Error:

Step 11/15 : RUN ./provision.sh
 ---> Running in 5159698cd65d
INFO [09-16|04:02:02.981] Maximum peer count                       ETH=50 LES=0 total=50
INFO [09-16|04:02:02.981] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=ledger vendor=11415 failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=21324 failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=4617  failcount=1 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=ledger vendor=11415 failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.982] Failed to enumerate USB devices          hub=trezor vendor=21324 failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
ERROR[09-16|04:02:02.983] Failed to enumerate USB devices          hub=trezor vendor=4617  failcount=2 err="failed to initialize libusb: libusb: unknown error [code -99]"
INFO [09-16|04:02:02.983] Allocated cache and file handles         database=/bsc/node1/geth/chaindata cache=16.00MiB handles=16
INFO [09-16|04:02:02.988] Writing custom genesis block 
INFO [09-16|04:02:02.990] Persisted trie from memory database      nodes=27 size=81.47KiB time=806.09µs gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-820.00B
INFO [09-16|04:02:02.990] Successfully wrote genesis state         database=chaindata                 hash=a0b4bf…36da90
INFO [09-16|04:02:02.990] Allocated cache and file handles         database=/bsc/node1/geth/lightchaindata cache=16.00MiB handles=16
INFO [09-16|04:02:02.996] Writing custom genesis block 
INFO [09-16|04:02:02.997] Persisted trie from memory database      nodes=27 size=81.47KiB time=271.457µs gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=-820.00B
INFO [09-16|04:02:02.997] Successfully wrote genesis state         database=lightchaindata                 hash=a0b4bf…36da90
Removing intermediate container 5159698cd65d
 ---> 115ccff85966
Step 12/15 : COPY ./entrypoint ${BSC_HOME}

This is fine, it shouldn't cause any issues.

simsonraj commented 3 years ago

@simsonraj

I did go mod download github.com/ethereum/go-ethereum but still got many errors like these with make under ./btp at this step:

Makefile:133: warning: overriding recipe for target 'test'
Makefile:50: warning: ignoring old recipe for target 'test'
[#] go build ./cmd/btpsimple
CGO_ENABLED=0 go build -tags "" -ldflags "-X 'main.version=1503db2-dirty' -X 'main.build=linux/amd64 tags()-2021-09-16-04:11:57'" -o ./bin/btpsimple ./cmd/btpsimple
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/core/state/database.go:23:2: missing go.sum entry for module providing package github.com/VictoriaMetrics/fastcache (imported by github.com/ethereum/go-ethereum/core/state); to add:
  go get github.com/ethereum/go-ethereum/core/state@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/crypto/signature_nocgo.go:28:2: missing go.sum entry for module providing package github.com/btcsuite/btcd/btcec (imported by github.com/ethereum/go-ethereum/crypto); to add:
  go get github.com/ethereum/go-ethereum/crypto@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/rpc/server.go:24:2: missing go.sum entry for module providing package github.com/deckarep/golang-set (imported by github.com/ethereum/go-ethereum/accounts/keystore); to add:
  go get github.com/ethereum/go-ethereum/accounts/keystore@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/consensus/ethash/ethash.go:36:2: missing go.sum entry for module providing package github.com/edsrzf/mmap-go (imported by github.com/ethereum/go-ethereum/consensus/ethash); to add:
  go get github.com/ethereum/go-ethereum/consensus/ethash@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/accounts/scwallet/hub.go:48:2: missing go.sum entry for module providing package github.com/gballet/go-libpcsclite (imported by github.com/ethereum/go-ethereum/accounts/scwallet); to add:
  go get github.com/ethereum/go-ethereum/accounts/scwallet@v1.10.2
../../go/pkg/mod/github.com/ethereum/go-ethereum@v1.10.2/log/handler.go:11:2: missing go.sum entry for module providing package github.com/go-stack/stack (imported by github.com/ethereum/go-ethereum/log); to add:
  go get github.com/ethereum/go-ethereum/log@v1.10.2

@tiendq, This is a weird one, never had this issue before, Can you please tell your OS and specs, I'll try to create a virtual env to see if this is happening from my side.

tiendq commented 3 years ago

@simsonraj

Ubuntu 20.04.2 Docker 20.10.8 docker-compose version 1.29.2

openjdk 11.0.11 2021-04-20 OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04) OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)

go version go1.17.1 linux/amd64

BTP branch: btp_web3labs

Run at cloned btp folder:

go mod download github.com/ethereum/go-ethereum make

simsonraj commented 3 years ago

@simsonraj

Ubuntu 20.04.2 Docker 20.10.8 docker-compose version 1.29.2

openjdk 11.0.11 2021-04-20 OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04) OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed mode, sharing)

go version go1.17.1 linux/amd64

BTP branch: btp_web3labs

Run at cloned btp folder:

go mod download github.com/ethereum/go-ethereum make

the commands you ran are in that particular order? It should just be make and the go mod download only when you have some issues. Thanks for the details, I'll check and get back to you.

tiendq commented 3 years ago

@simsonraj

I did run both of them. I saw it was missing from last time Sang did it so I thought it was a required step.

simsonraj commented 3 years ago

@tiendq ,

I tried a fresh setup to figure out this, I faced an issue similar to yours when my go version was 1.17.X. When tried with 1.16.X , I was able to fix make with "go mod download..." & then "go mod tidy", then running "make".

Please try with the downgraded Go 1.16 & let us know, we have updated the guide for the same.

tiendq commented 3 years ago

@tiendq ,

I tried a fresh setup to figure out this, I faced an issue similar to yours when my go version was 1.17.X. When tried with 1.16.X , I was able to fix make with "go mod download..." & then "go mod tidy", then running "make".

Please try with the downgraded Go 1.16 & let us know, we have updated the guide for the same.

@simsonraj

I re-installed Go 1.16.8 (go1.16.8.linux-amd64.tar.gz), it worked.

But I got an error with next step make dist-javascore, according to last log lines (attached log) there were no contracts created.

/home/ubuntu/testnet/btp is cloned repo where I run commands.

invalid output path: directory "/home/ubuntu/testnet/btp/build/contracts" does not exist
invalid output path: directory "/home/ubuntu/testnet/btp/build/contracts" does not exist
invalid output path: directory "/home/ubuntu/testnet/btp/build/contracts" does not exist
invalid output path: directory "/home/ubuntu/testnet/btp/build/contracts" does not exist

dist-javascore.log

simsonraj commented 3 years ago

@tiendq Sorry, fixed this issue, it wasn't able to create the folder, can you please try again with the latest code. Also please clean your existing images that were created 

docker-compose down
rm -rf work/*
docker rmi -f icon-bsc_btp-icon
docker rm -f javascore-dist
docker rmi -f btpsimple
docker rmi -f btp/javascore
tiendq commented 3 years ago

@tiendq Sorry, fixed this issue, it wasn't able to create the folder, can you please try again with the latest code. Also please clean your existing images that were created

docker-compose down
rm -rf work/*
docker rmi -f icon-bsc_btp-icon
docker rm -f javascore-dist
docker rmi -f btpsimple
docker rmi -f btp/javascore

It's successful now, thank you.

Successfully built 5614978520e7
Successfully tagged btp/javascore:latest
2029ce099d9c43360412a92ac6af656bd880099cfb341690e9a874ce59c2861d
tiendq commented 3 years ago

@simsonraj I got new issue below with docker-compose up

ubuntu@ip-172-31-19-19:~/testnet/btp/devnet/docker/icon-bsc$ docker-compose up
Creating network "icon-bsc_default" with the default driver
Creating binancesmartchain ... done
Creating goloop            ... done
Creating btp-icon          ... done

ERROR: for btp-bsc  Container "5eafacd5d129" is unhealthy.
ERROR: Encountered errors while bringing up the project.

There was an error with 5eafacd5d129 icon-bsc_btp-icon btp-icon container. I tried to start manually but it still failed.

docker container logs 5eafacd5d129 told me below log:

/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
simsonraj commented 3 years ago

hi @

@simsonraj I got new issue below with docker-compose up

ubuntu@ip-172-31-19-19:~/testnet/btp/devnet/docker/icon-bsc$ docker-compose up
Creating network "icon-bsc_default" with the default driver
Creating binancesmartchain ... done
Creating goloop            ... done
Creating btp-icon          ... done

ERROR: for btp-bsc  Container "5eafacd5d129" is unhealthy.
ERROR: Encountered errors while bringing up the project.

There was an error with 5eafacd5d129 icon-bsc_btp-icon btp-icon container. I tried to start manually but it still failed.

docker container logs 5eafacd5d129 told me below log:

/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied
/btpsimple/bin/entrypoint.sh: /btpsimple/bin/provision.sh: line 427: wait-for-it.sh: Permission denied

Hi @tiendq , From the logs above Creating btp-icon ... done this is the container you need to initiate token transfer, looks like the btp-bsc container failed, that is normal for now until we push the other side working too.

Also I assume you restarted btp-icon having it started already that's why you get this issue.

tiendq commented 3 years ago

@simsonraj

No, there must be something wrong here. Problem is btp-icon container was only created but never initialized. Even in my first command docker-compose up it exited immediately.

Provisioning has never started, I believe Permission denied error stopped it. (there are 4 lines since I tried to start it several times).

Screen Shot 2021-09-20 at 14 23 30
simsonraj commented 3 years ago

@simsonraj

No, there must be something wrong here. Problem is btp-icon container was only created but never initialized. Even in my first command docker-compose up it exited immediately.

Provisioning has never started, I believe Permission denied error stopped it. (there are 4 lines since I tried to start it several times).

Screen Shot 2021-09-20 at 14 23 30

This is weird, it worked for me when I last gave you update, I still added a line in the script to fix the issue, Please try now.

tiendq commented 3 years ago

@simsonraj

I need to pull new update, don't I?

And please note what step I need to re-start since I guess there're new/modified files under the btp folder, and some containers might need to be rebuilt etc.

simsonraj commented 3 years ago

Yes, please pull the latest. You can just use the ./build.sh that should take care of it. if you still need a clean setup, you can do manually

docker-compose down
rm -rf work/*
docker rmi -f icon-bsc_btp-icon
docker rm -f javascore-dist
docker rmi -f btpsimple
docker rmi -f btp/javascore
tiendq commented 3 years ago

Yes, please pull the latest. You can just use the ./build.sh that should take care of it. if you still need a clean setup, you can do manually

docker-compose down
rm -rf work/*
docker rmi -f icon-bsc_btp-icon
docker rm -f javascore-dist
docker rmi -f btpsimple
docker rmi -f btp/javascore

I restarted with ./build.sh, docker-compose up took about 15-20 minutes to create containers (not exit immediately) but finally it still exited with an error with btp-icon as before. I attached log from this container, you could see at least there was an error at the end.

test1.log.zip

simsonraj commented 3 years ago

@tiendq , We aren't sure why this happens for you, To try and replicate this issue, I created a new VM, with a similar setup as yours, and I did not face this issue. Please check the attached screenshot and video recording of the entire flow, including the token transfer. My only guess at this point is some temporary network issue. I suggest you try it one more time and see if the same issue persists.

image

https://user-images.githubusercontent.com/5087697/133988148-21b5acf9-624d-4097-a371-c9980886d7c1.mp4

https://user-images.githubusercontent.com/5087697/133988159-a1e2d3c0-7b8f-4094-be35-b5c813df7226.mp4

tiendq commented 3 years ago

@simsonraj

Thanks, I'll try it again. Our server is on the cloud, provide me your SSH key if you want to check it yourself.

tiendq commented 3 years ago

@simsonraj

Finally, it works! And I tried to make a transfer by following your commands in the above screencast.

  1. Transferring wasn't success, in last getBalance balance is unchanged. 
    # getBalance 0x70e789d2f5d469ea30e0525dbfdd5515d6ead30d
# User Balance: 9999999999999999999900

Here is result of source /btpsimple/bin/transfer_eth.sh:

alice.ks.json
Transfering 0xa to: hxd7d038648b7c32268b0a895419a5640597963894 from: hx1db0d2f5746f19a735d8fbb38197b3d691c38c80
{ "to": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "cumulativeStepUsed": "0x3919", "stepUsed": "0x3919", "stepPrice": "0x0", "eventLogs": [ { "scoreAddress": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "indexed": [ "Transfer(Address,Address,int,bytes)", "hx1db0d2f5746f19a735d8fbb38197b3d691c38c80", "hxd7d038648b7c32268b0a895419a5640597963894", "0xa" ], "data": [ "0x" ] } ], "logsBloom": "0x00008000000000000020002000000000000000000000000000000000000100000000000000000000000000000000022000000000000000000000000000000000000000120000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000200000000000000000000001000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001000000000000000000000000000000010000000000000000000000000000000000000000000000000000", "status": "0x1", "blockHash": "0x756598bc5433308da50f7072d0f9c7159a366e80db4412b6ab53f2375b2286ee", "blockHeight": "0x99f", "txIndex": "0x0", "txHash": "0x9c8b83929e552cee4ceabb09c32b96c4074187c1a97babd64ce07556ba94d11e" }
alice.ks.json
Transfering 0xa to: cxf83cb683351eac91a19f12f760f8c9739abc8f33 from: hxd7d038648b7c32268b0a895419a5640597963894
{ "to": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "cumulativeStepUsed": "0x712b", "stepUsed": "0x712b", "stepPrice": "0x0", "eventLogs": [ { "scoreAddress": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "indexed": [ "Transfer(Address,Address,int,bytes)", "hxd7d038648b7c32268b0a895419a5640597963894", "cxf83cb683351eac91a19f12f760f8c9739abc8f33", "0xa" ], "data": [ "0x" ] } ], "logsBloom": "0x00000000000000000020002000000000000000000000000000000000000100000000000000000000000000000000002000040000000000000000000000000000000000420000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008800000001000000000000000000000000000000010000000000000000000000000000000000000000000000000000", "status": "0x1", "blockHash": "0x91c043e6f73d42ece19a74f9c0d8e0a5139192031e66f022062a12fdfdecec07", "blockHeight": "0x9a1", "txIndex": "0x0", "txHash": "0x2e6923f51da3748250fd6b7cdef63152f2aa9b09c7f150f6e4e95b50f5843e38" }
Balance of user hxd7d038648b7c32268b0a895419a5640597963894
{
  "locked": "0x0",
  "refundable": "0x0",
  "usable": "0xa"
}
alice.ks.json

  1. I saw only Alice, who are sender and receiver? and what are source and destination networks?

  2. Could you provide me a set of transfer scripts/commands where I could run from host server (not inside a container)? I had these scripts for current testnet (ICON & Moonbeam) so can run a quick test anytime.

simsonraj commented 3 years ago

@simsonraj

Finally, it works! And I tried to make a transfer by following your commands in the above screencast.

  1. Transferring wasn't success, in last getBalance balance is unchanged.
# getBalance 0x70e789d2f5d469ea30e0525dbfdd5515d6ead30d
# User Balance: 9999999999999999999900

Here is result of source /btpsimple/bin/transfer_eth.sh:

alice.ks.json
Transfering 0xa to: hxd7d038648b7c32268b0a895419a5640597963894 from: hx1db0d2f5746f19a735d8fbb38197b3d691c38c80
{ "to": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "cumulativeStepUsed": "0x3919", "stepUsed": "0x3919", "stepPrice": "0x0", "eventLogs": [ { "scoreAddress": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "indexed": [ "Transfer(Address,Address,int,bytes)", "hx1db0d2f5746f19a735d8fbb38197b3d691c38c80", "hxd7d038648b7c32268b0a895419a5640597963894", "0xa" ], "data": [ "0x" ] } ], "logsBloom": "0x00008000000000000020002000000000000000000000000000000000000100000000000000000000000000000000022000000000000000000000000000000000000000120000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000200000000000000000000001000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000001000000000000000000000000000000010000000000000000000000000000000000000000000000000000", "status": "0x1", "blockHash": "0x756598bc5433308da50f7072d0f9c7159a366e80db4412b6ab53f2375b2286ee", "blockHeight": "0x99f", "txIndex": "0x0", "txHash": "0x9c8b83929e552cee4ceabb09c32b96c4074187c1a97babd64ce07556ba94d11e" }
alice.ks.json
Transfering 0xa to: cxf83cb683351eac91a19f12f760f8c9739abc8f33 from: hxd7d038648b7c32268b0a895419a5640597963894
{ "to": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "cumulativeStepUsed": "0x712b", "stepUsed": "0x712b", "stepPrice": "0x0", "eventLogs": [ { "scoreAddress": "cxdf1b5a0f39254436ef954208bceb958352e130fa", "indexed": [ "Transfer(Address,Address,int,bytes)", "hxd7d038648b7c32268b0a895419a5640597963894", "cxf83cb683351eac91a19f12f760f8c9739abc8f33", "0xa" ], "data": [ "0x" ] } ], "logsBloom": "0x00000000000000000020002000000000000000000000000000000000000100000000000000000000000000000000002000040000000000000000000000000000000000420000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000110000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008800000001000000000000000000000000000000010000000000000000000000000000000000000000000000000000", "status": "0x1", "blockHash": "0x91c043e6f73d42ece19a74f9c0d8e0a5139192031e66f022062a12fdfdecec07", "blockHeight": "0x9a1", "txIndex": "0x0", "txHash": "0x2e6923f51da3748250fd6b7cdef63152f2aa9b09c7f150f6e4e95b50f5843e38" }
Balance of user hxd7d038648b7c32268b0a895419a5640597963894
{
  "locked": "0x0",
  "refundable": "0x0",
  "usable": "0xa"
}
alice.ks.json
  1. I saw only Alice, who are sender and receiver? and what are source and destination networks?
  2. Could you provide me a set of transfer scripts/commands where I could run from host server (not inside a container)? I had these scripts for current testnet (ICON & Moonbeam) so can run a quick test anytime.

@tiendq ,

  1. The transaction looks fine to me, it might take a couple of seconds or even a minute sometimes to initially sync. Please try retrieving the transfer balance after giving it some time.
  2. Alice is the sender from ICON network(source), bsc.ks.json is the receiver on the Binance smart chain network(destination) . this is also mentioned in the token-transfer-guide at the end.
  3. Sure, I'll work on this, please give me a day or two. I'll get back to you on this.
tiendq commented 3 years ago

@simsonraj

  1. I'm afraid transaction wasn't success, I still see same balance. Where can I look at to see if there was any error?

  2. Sorry, I missed the transfer guide, I just followed your commands above.

Thank you,

simsonraj commented 3 years ago

@simsonraj

  1. I'm afraid transaction wasn't success, I still see same balance. Where can I look at to see if there was any error?
  2. Sorry, I missed the transfer guide, I just followed your commands above.

Thank you,

  1. can you find this file tx.transfer under the work folder? this should contain the transaction for the transfer. Btw did you start the network again to test the transfer?
tiendq commented 3 years ago
  1. can you find this file tx.transfer under the work folder? this should contain the transaction for the transfer. Btw did you start the network again to test the transfer?

I saw it, it has transaction information and it's success. But in my experience even transaction is success on sender side, there's often something wrong in other internal transactions with relay and/or receiver side.

simsonraj commented 3 years ago
  1. can you find this file tx.transfer under the work folder? this should contain the transaction for the transfer. Btw did you start the network again to test the transfer?

I saw it, it has transaction information and it's success. But in my experience even transaction is success on sender side, there's often something wrong in other internal transactions with relay and/or receiver side.

Agreed, when you perform, transfer transaction, can you please keep an eye on the btp-icon container logs, and see if you find any errors in the timeframe

tiendq commented 3 years ago

@simsonraj

I think I have faced a new issue.

  1. I couldn't file anything with test transfer in btp-icon logs!

  2. I stopped all containers yesterday and restart them today with docker-compose start but btp-icon and btp-bsc exited, below is their last log, they show an error. I tried start them manually but no luck.

btp-icon

D|07:36:44.542297|70E7|0x97|logrus|entry.go:314 addRelayMessage rms:77 bu:2469 ~ 5223
D|07:36:45.543356|70E7|0x97|logrus|entry.go:314 addRelayMessage rms:77 bu:2469 ~ 5224
D|07:36:46.550327|70E7|0x97|logrus|entry.go:314 addRelayMessage rms:77 bu:2469 ~ 5225
I|06:56:27.764690|----|-|logrus|entry.go:359   ____ _____ ____    ____      _
I|06:56:27.764828|----|-|logrus|entry.go:359  | __ )_   _|  _ \  |  _ \ ___| | __ _ _   _
I|06:56:27.764843|----|-|logrus|entry.go:359  |  _ \ | | | |_) | | |_) / _ \ |/ _` | | | |
I|06:56:27.764854|----|-|logrus|entry.go:359  | |_) || | |  __/  |  _ <  __/ | (_| | |_| |
I|06:56:27.764865|----|-|logrus|entry.go:359  |____/ |_| |_|     |_| \_\___|_|\__,_|\__, |
I|06:56:27.764894|----|-|logrus|entry.go:359                                        |___/ 
I|06:56:27.764914|----|-|logrus|entry.go:314 Version : e1f805d-dirty
I|06:56:27.764932|----|-|logrus|entry.go:314 Build   : linux/amd64 tags()-2021-09-21-04:58:52
D|06:56:29.974028|70E7|-|logrus|entry.go:359 LogForwarderConfig vendor and address is empty string, will be ignore
D|06:56:29.974220|70E7|-|logrus|entry.go:359 /btpsimple/config/icon.config.json ../data/btp_icon_bsc
F|06:56:29.974246|70E7|-|logrus|entry.go:314 Sender not supported for chain %!(EXTRA string=)
I|06:58:09.475257|----|-|logrus|entry.go:359   ____ _____ ____    ____      _
I|06:58:09.475326|----|-|logrus|entry.go:359  | __ )_   _|  _ \  |  _ \ ___| | __ _ _   _
I|06:58:09.475338|----|-|logrus|entry.go:359  |  _ \ | | | |_) | | |_) / _ \ |/ _` | | | |
I|06:58:09.475349|----|-|logrus|entry.go:359  | |_) || | |  __/  |  _ <  __/ | (_| | |_| |
I|06:58:09.475414|----|-|logrus|entry.go:359  |____/ |_| |_|     |_| \_\___|_|\__,_|\__, |
I|06:58:09.475442|----|-|logrus|entry.go:359                                        |___/ 
I|06:58:09.475499|----|-|logrus|entry.go:314 Version : e1f805d-dirty
I|06:58:09.475535|----|-|logrus|entry.go:314 Build   : linux/amd64 tags()-2021-09-21-04:58:52
D|06:58:10.418747|70E7|-|logrus|entry.go:359 LogForwarderConfig vendor and address is empty string, will be ignore
D|06:58:10.418865|70E7|-|logrus|entry.go:359 /btpsimple/config/icon.config.json ../data/btp_icon_bsc
F|06:58:10.418891|70E7|-|logrus|entry.go:314 Sender not supported for chain %!(EXTRA string=)

btp-bsc

I|06:17:06.731832|----|-|logrus|entry.go:359   ____ _____ ____    ____      _
I|06:17:06.731902|----|-|logrus|entry.go:359  | __ )_   _|  _ \  |  _ \ ___| | __ _ _   _
I|06:17:06.731913|----|-|logrus|entry.go:359  |  _ \ | | | |_) | | |_) / _ \ |/ _` | | | |
I|06:17:06.731925|----|-|logrus|entry.go:359  | |_) || | |  __/  |  _ <  __/ | (_| | |_| |
I|06:17:06.731936|----|-|logrus|entry.go:359  |____/ |_| |_|     |_| \_\___|_|\__,_|\__, |
I|06:17:06.731952|----|-|logrus|entry.go:359                                        |___/ 
I|06:17:06.731966|----|-|logrus|entry.go:314 Version : e1f805d-dirty
I|06:17:06.731980|----|-|logrus|entry.go:314 Build   : linux/amd64 tags()-2021-09-21-04:58:52
D|06:17:06.938235|1db0|-|logrus|entry.go:359 LogForwarderConfig vendor and address is empty string, will be ignore
D|06:17:06.938516|1db0|-|logrus|entry.go:359 /btpsimple/config/bsc.config.json ../data/btp_icon_bsc
D|06:17:06.941115|1db0|0x5d3cec|logrus|entry.go:359 open database /btpsimple/data/btp_icon_bsc/0x5d3cec.icon
D|06:17:07.060393|1db0|0x5d3cec|logrus|entry.go:314 _init height:2, dst(btp://0x5d3cec.icon/cx2c114e40419d5be7fd7f29490361bdf2408480fd, height:2, seq:0, last:2), receive:2
D|06:17:07.060465|1db0|-|logrus|entry.go:314 ReceiveLoop connected
D|06:17:07.060439|1db0|0x5d3cec|logrus|entry.go:359 start relayLoop
D|06:17:07.062854|1db0|-|logrus|entry.go:314 MonitorBlock WSEvent 172.19.0.5:46832 WSEventInit
D|06:17:07.062892|1db0|-|logrus|entry.go:314 MonitorLoop connected 172.19.0.5:46832
D|06:17:07.062907|1db0|0x5d3cec|logrus|entry.go:314 Connect MonitorLoop
I|06:57:45.138453|----|-|logrus|entry.go:359   ____ _____ ____    ____      _
I|06:57:45.138529|----|-|logrus|entry.go:359  | __ )_   _|  _ \  |  _ \ ___| | __ _ _   _
I|06:57:45.138542|----|-|logrus|entry.go:359  |  _ \ | | | |_) | | |_) / _ \ |/ _` | | | |
I|06:57:45.138579|----|-|logrus|entry.go:359  | |_) || | |  __/  |  _ <  __/ | (_| | |_| |
I|06:57:45.138590|----|-|logrus|entry.go:359  |____/ |_| |_|     |_| \_\___|_|\__,_|\__, |
I|06:57:45.138604|----|-|logrus|entry.go:359                                        |___/ 
I|06:57:45.138614|----|-|logrus|entry.go:314 Version : e1f805d-dirty
I|06:57:45.138655|----|-|logrus|entry.go:314 Build   : linux/amd64 tags()-2021-09-21-04:58:52
D|06:57:45.355988|1db0|-|logrus|entry.go:359 LogForwarderConfig vendor and address is empty string, will be ignore
D|06:57:45.356127|1db0|-|logrus|entry.go:359 /btpsimple/config/bsc.config.json ../data/btp_icon_bsc
F|06:57:45.356154|1db0|-|logrus|entry.go:314 Sender not supported for chain %!(EXTRA string=)
simsonraj commented 3 years ago

@tiendq, this happens when we restart the stopped containers as it is, Please try with ./build.sh to remove the work folder & clear the containers, and then docker-compose up -d to bring it back up again.

tiendq commented 3 years ago

@tiendq, this happens when we restart the stopped containers as it is, Please try with ./build.sh to remove the work folder & clear the containers, and then docker-compose up -d to bring it back up again.

I deployed it successfully again, getBalance still shows result at last time (unchanged balance). Then I checked the accounts as below. Is it expected result?

goloop rpc balance hxf6f258fdf3d090bce14f0073f9e5a5587384cc0c --uri http://localhost:9080/api/v3/icon
# 0 -> Alice, it seems correct.
eth address:balance 0x70e789d2f5d469ea30e0525dbfdd5515d6ead30d
# 9999999.178615693 -> bsc.ks.json, not sure

btp-icon.log

simsonraj commented 3 years ago

@tiendq, this happens when we restart the stopped containers as it is, Please try with ./build.sh to remove the work folder & clear the containers, and then docker-compose up -d to bring it back up again.

I deployed it successfully again, getBalance still shows result at last time (unchanged balance). Then I checked the accounts as below. Is it expected result?

goloop rpc balance hxf6f258fdf3d090bce14f0073f9e5a5587384cc0c --uri http://localhost:9080/api/v3/icon
# 0 -> Alice, it seems correct.
eth address:balance 0x70e789d2f5d469ea30e0525dbfdd5515d6ead30d
# 9999999.178615693 -> bsc.ks.json, not sure

btp-icon.log

Since we are working on the token transfer scenario, we don't care about the native balances of the users, we need to check the token balances, IRC2 balance of Alice in ICON, & ERC20 balance of bsc.ks.json on Binance. Precisely what the getBalance method does.

But as far as the transfer, I'm still not sure what's happening there, I'll get back to you on this after analyzing it a bit. Can I ask you to try this scenario, if possible, to try the token transfer as soon as the btp-icon container is up, without waiting, to investigate the possibility of messages getting out of sync?

tiendq commented 3 years ago

@simsonraj

I forgot it isn't native coin so need to check against contract.

FYI, I tried the transfer right after containers all up.

simsonraj commented 3 years ago

Thanks @tiendq , So I assume the result is the same, we are also trying to figure out from our end, iI'll get back to you on this.