Migrate dev/prod environment to wns from registry

dboreham commented 4 years ago

Deploy WNS servers to replace the current registry servers, fix up whatever needs to be changed in our applications/tooling/bots to interop with WNS.

ashwinphatak commented 4 years ago

[x] Setup 4 node WNS dev and prod testnets
[x] Update infra doc for WNS testnets
[x] Check if any (ARM/arch) issues compiling/running WNS on wirebox
[x] Hotfix ARM byte alignment issue on wirebox
[x] Rebase feature-wns-support branch in all changed repos on top of respective master branches and retest everything
[x] Update launchpad scripts and demo docs to use WNS commands, URL/paths.
[x] Merge feature-wns-support branches for wireline-core, registry-client, registry-cli, botkit, launchpad into master
[x] Bump versions on repos/master / use updated versions in other repos
- [x] Bump minor version on wireline-core
- [x] Bump minor version on registry-client
- [x] Bump minor version on registry-cli after upgrading to latest of registry-client
- [x] Bump minor version on botkit
- [x] Bump minor versions on chess, shogi and logbot-template after upgrading to latest of botkit
[x] Testing: Play chess between two collabs on https://launchpad.dev.wireline.ninja/
[x] Testing: All on localhost with latest wire-cli: Play from LP with chess repo bot, shogi repo bot, devcon shogi demo
[x] New wirebox image with WNS and other changes merged (with bots running)
[x] Redeploy and register cloud bots with dev/prod WNS testnets
[x] Testing: https://launchpad.wireline.ninja/, https://launchpad.dev.wireline.ninja/#/, wirebox, demo using cloud & wirebox bots, retry devcon demo with WNS. Test @wirelineio and @example chess and shogi pads/bots.
[x] Spin down old registry dev and prod testnets
[x] Archive registry GitHub repository

ashwinphatak commented 4 years ago

Note: Catalina issue trying to build latest launchpad.

warning "workspace-aggregator-540e98c8-748a-40cf-85b2-c60f4a761652 > @wirelineio/app > @apollo/react-components > @apollo/react-hooks@3.1.3" has unmet peer dependency "@types/react@^16.8.0".
warning "wrtc@0.4.2" is missing a bundled dependency "node-pre-gyp". This should be reported to the package maintainer.
[4/4] 🔨  Building fresh packages...
[15/23] ⠂ canvas
[18/23] ⠂ puppeteer
[3/23] ⠂ scrypt
[14/23] ⠂ wrtc
warning Error running install script for optional dependency: "/Users/ashwinp/projects/wireline/launchpad/node_modules/scrypt: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: /Users/ashwinp/projects/wireline/launchpad/node_modules/scrypt
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@5.0.5
gyp info using node@11.15.0 | darwin | x64
gyp info find Python using Python version 2.7.16 found at \"/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python\"
gyp info spawn /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

ashwinphatak commented 4 years ago

https://github.com/nodejs/node-gyp/issues/1927#issuecomment-548234995 solved the above issue for me.

ashwinphatak commented 4 years ago

Current bots don't work with latest LP after recent framework upgrades over the weekend. Need to investigate.

ashwinphatak commented 4 years ago

Botkit framework upgrade to v0.2.23 solved the issue.

ashwinphatak commented 4 years ago

Built WNS 'manually' on my wirebox for testing purposes. Server starts and is creating blocks but get this error on broadcasting a set record transaction.

I[2019-11-04|13:23:15.004] Executed block                               module=state height=91 validTxs=0 invalidTxs=0
I[2019-11-04|13:23:15.030] Committed state                              module=state height=91 txs=0 appHash=F0C1582A986C8DB55EA0B3AADF4802104BFE3DB78C4AF1197DD155F90EAC3D93
I[2019-11-04|13:23:20.229] Executed block                               module=state height=92 validTxs=0 invalidTxs=0
I[2019-11-04|13:23:20.253] Committed state                              module=state height=92 txs=0 appHash=9E9A5370976181AA219EE2AD3501C82CD2935989DD66172A163E642A157AEBE8
E[2019-11-04|13:23:21.518] Panic in RPC HTTP handler                    module=rpc-server err="runtime error: invalid memory address or nil pointer dereference" stack="goroutine 322 [running]:\nruntime/debug.Stack(0x34ed730, 0xacedb0, 0x16aff70)\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x78\ngithub.com/tendermint/tendermint/rpc/lib/server.RecoverAndLogHandler.func1.1(0x554e170, 0xf67918, 0x28e18d0, 0x5ed657e2, 0xbf682772, 0x34e8f622, 0x10, 0x16bda10, 0x5529c80)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/lib/server/http_server.go:161 +0x2e8\npanic(0xacedb0, 0x16aff70)\n\t/usr/local/go/src/runtime/panic.go:679 +0x194\nruntime/internal/atomic.goLoad64(0x310ff54, 0x0, 0x1cebc)\n\t/usr/local/go/src/runtime/internal/atomic/atomic_arm.go:131 +0x1c\ngithub.com/tendermint/tendermint/mempool.(*CListMempool).TxsBytes(...)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/mempool/clist_mempool.go:171\ngithub.com/tendermint/tendermint/mempool.(*CListMempool).CheckTxWithInfo(0x310ff00, 0x4f91380, 0x189, 0x189, 0x5506ee0, 0xac0000, 0x0, 0x0)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/mempool/clist_mempool.go:222 +0x98\ngithub.com/tendermint/tendermint/mempool.(*CListMempool).CheckTx(0x310ff00, 0x4f91380, 0x189, 0x189, 0x5506ee0, 0xad7d90, 0x1)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/mempool/clist_mempool.go:212 +0x44\ngithub.com/tendermint/tendermint/rpc/core.BroadcastTxSync(0x554e190, 0x4f91380, 0x189, 0x189, 0x0, 0x0, 0x0)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/core/mempool.go:138 +0x94\nreflect.Value.call(0xa940f8, 0xdb464c, 0x13, 0xbb59ef, 0x4, 0x5533fe0, 0x2, 0x2, 0x2, 0xc, ...)\n\t/usr/local/go/src/reflect/value.go:460 +0x49c\nreflect.Value.Call(0xa940f8, 0xdb464c, 0x13, 0x5533fe0, 0x2, 0x2, 0x1, 0x1, 0x0)\n\t/usr/local/go/src/reflect/value.go:321 +0x78\ngithub.com/tendermint/tendermint/rpc/lib/server.makeJSONRPCHandler.func1(0xf659f8, 0x554e170, 0x5529c80)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/lib/server/handlers.go:157 +0x678\ngithub.com/tendermint/tendermint/rpc/lib/server.handleInvalidJSONRPCPaths.func1(0xf659f8, 0x554e170, 0x5529c80)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/lib/server/handlers.go:181 +0x88\nnet/http.HandlerFunc.ServeHTTP(0x3242a28, 0xf659f8, 0x554e170, 0x5529c80)\n\t/usr/local/go/src/net/http/server.go:2007 +0x34\nnet/http.(*ServeMux).ServeHTTP(0x292c810, 0xf659f8, 0x554e170, 0x5529c80)\n\t/usr/local/go/src/net/http/server.go:2387 +0x188\ngithub.com/tendermint/tendermint/rpc/lib/server.maxBytesHandler.ServeHTTP(0xf5dd40, 0x292c810, 0xf4240, 0x0, 0xf659f8, 0x554e170, 0x5529c80)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/lib/server/http_server.go:206 +0xc4\ngithub.com/tendermint/tendermint/rpc/lib/server.RecoverAndLogHandler.func1(0xf65d78, 0x4f4d5f0, 0x5529c80)\n\t/home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/rpc/lib/server/http_server.go:179 +0x284\nnet/http.HandlerFunc.ServeHTTP(0x325a2e0, 0xf65d78, 0x4f4d5f0, 0x5529c80)\n\t/usr/local/go/src/net/http/server.go:2007 +0x34\nnet/http.serverHandler.ServeHTTP(0x301c630, 0xf65d78, 0x4f4d5f0, 0x5529c80)\n\t/usr/local/go/src/net/http/server.go:2802 +0x88\nnet/http.(*conn).serve(0x53c6c60, 0xf673b8, 0x541e580)\n\t/usr/local/go/src/net/http/server.go:1890 +0x7e0\ncreated by net/http.(*Server).Serve\n\t/usr/local/go/src/net/http/server.go:2927 +0x2f0\n"
I[2019-11-04|13:23:25.458] Executed block                               module=state height=93 validTxs=0 invalidTxs=0
I[2019-11-04|13:23:25.488] Committed state                              module=state height=93 txs=0 appHash=7F37A10B531F20C94BB86B693AEF49355605FBDAD367F8FF8428B246F080AF0F
I[2019-11-04|13:23:30.877] Executed block                               module=state height=94 validTxs=0 invalidTxs=0

ashwinphatak commented 4 years ago

To repro:

cd /Users/ashwinp/projects/wireline/wns/x/nameservice/examples
wnscli tx nameservice set protocol.yml --from root --node tcp://wirebox.local:26657

ashwinphatak commented 4 years ago

Looks like ARM byte alignment issues: https://github.com/golang/go/issues/23345

Also see https://golang.org/pkg/sync/atomic/#pkg-note-BUG

ashwinphatak commented 4 years ago

Patching tendermint and rebuilding seems to make WNS work.

sudo patch /home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/mempool/clist_mempool.go ./golang-arm-atomic-64bit.patch
patching file /home/pi/go/pkg/mod/github.com/tendermint/tendermint@v0.32.2/mempool/clist_mempool.go

make install

Patch file:

golang-arm-atomic-64bit.patch:

--- clist_mempool.go    2019-11-05 06:36:39.468195935 +0000
+++ clist_mempool_fixed.go  2019-11-05 06:38:32.657729361 +0000
@@ -29,6 +29,10 @@
 // mempool uses a concurrent list structure for storing transactions that can
 // be efficiently accessed by multiple concurrent readers.
 type CListMempool struct {
+   // Atomic 64-bit integers. Hotfix for https://github.com/golang/go/issues/23345.
+   height     int64 // the last block Update()'d to
+   txsBytes   int64 // total size of mempool, in bytes
+
    config *cfg.MempoolConfig

    proxyMtx     sync.Mutex
@@ -52,9 +56,7 @@
    txsMap sync.Map

    // Atomic integers
-   height     int64 // the last block Update()'d to
    rechecking int32 // for re-checking filtered txs on Update()
-   txsBytes   int64 // total size of mempool, in bytes

    // Keep a cache of already-seen txs.
    // This reduces the pressure on the proxyApp.

wirelineio / wns

Migrate dev/prod environment to wns from registry #59