Plugin Development : Cosmos (V1)

heejin-github commented 2 years ago

Parents: #241
Related: #225

Checklist

[X] New Feature for the Service/Plugin (Vatz)
[ ] Enhancement (Vatz)
[ ] others(etc. e.g, documentation, project policy management)

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. such as Ex. I'm always frustrated

make cosmos plugin as an official plugin.

Describe the solution you'd like| Ex

A clear and concise description of what you want to happen

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you have considered.

Additional context or comment

Add any other context or screenshots about the feature request here.

Child Issues

Phase 1.

[x] #243
[x] #244
[x] #245

Phase 2.
[x] #265
[x] #266
[x] #267
[x] #268
[x] #269
[x] #270
[x] #288
[x] #294
[x] #303
[x] #304
[x] #326

heejin-github commented 2 years ago

1st phase

we already make some plugins for cosmos-sdk/tendermint based project and linux systems. trying to make first version of cosmos plugin with existed plugins, method, or systems in 1st phase.

Create new repo for vatz-plugin-cosmos

Collect developed plugins into vatz-plugin-cosmos

exist one : up, block-height, hwstat(cpu, mem, disk)

need develop : peer count

All plugins will be based on the Vatz SDK and have its own module.

Build all plugins with make

Run and manage plugins with pm2

Testing on cosmos mainnet sentry machine or any other cosmos-sdk based testnet nodes.

2nd phase

Find improvements and newly developments, if needed.

Check any problem to monitor cosmos node with current plugins.
Check if any further monitoring items required.
Check for unnecessary monitoring items in current plugins.
Find enhancement for Build and Run.
Decide whether cosmos plugin integrate into Vatz and CLI interaction
Discuss about any action items or requirements for the next phase.

@xellos00 @meetrick @Choi-Jinhong @skonhwang any feedback is welcome! Let's start to develop cosmos plugin.

heejin-github commented 2 years ago

plugin list

./vatz-plugin-cosmos
└── plugin
    ├── chainstat
    │   ├── block_height
    │   ├── peer_count
    │   └── up
    └── hwstat
        ├── cpu
        ├── disk
        └── mem

heejin-github commented 2 years ago

CI

Build and Test with github action https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go

xellos00 commented 2 years ago

plugin list

./vatz-plugin-cosmos
└── plugin
    ├── chainstat
    │   ├── block_height
    │   ├── peer_count
    │   └── up
    └── hwstat
        ├── cpu
        ├── disk
        └── mem

I prefer to separate that for checking hardwares because it's going to be used for everyone.

heejin-github commented 2 years ago

plugin list

./vatz-plugin-cosmos
└── plugin
    ├── chainstat
    │   ├── block_height
    │   ├── peer_count
    │   └── up
    └── hwstat
        ├── cpu
        ├── disk
        └── mem

I prefer to separate that for checking hardwares because it's going to be used for everyone.

plugin separated to vatz-plugin-cosmoshub and vatz-plugin-sysutil as we discussed from #225

# tree ./vatz-plugin-sysutil
./vatz-plugin-sysutil
├── plugins
│   ├── cpu_monitor
│   │   ├── go.mod
│   │   ├── go.sum
│   │   └── main.go
│   ├── disk_monitor
│   │   ├── go.mod
│   │   ├── go.sum
│   │   └── main.go
│   └── mem_monitor
│       ├── go.mod
│       ├── go.sum
│       └── main.go
└── README.md

# tree ./vatz-plugin-cosmoshub
./vatz-plugin-cosmoshub
├── plugins
│   ├── cosmos-sdk-blocksync
│   │   ├── main.go
│   │   ├── main_test.go
│   │   ├── Makefile
│   │   ├── mocks
│   │   │   ├── Collector.go
│   │   │   └── Estimator.go
│   │   ├── policy
│   │   │   ├── policy.go
│   │   │   └── policy_test.go
│   │   └── status
│   │       ├── status.go
│   │       └── status_test.go
│   ├── is_alived
│   │   ├── go.mod
│   │   ├── go.sum
│   │   └── main.go
│   └── peer_count
│       ├── go.mod
│       ├── go.sum
│       └── main.go
├── README.md
└── rpc
    └── cosmos
        ├── api.go
        ├── api_test.go
        ├── fixtures.go
        └── mocks
            └── Client.go

heejin-github commented 2 years ago

2022-10-11 `vatz-plugin-cosmoshub` Phase 1 Development

PR list

Summary

Collect developed plugins and revised it with RPC API call.
Can execute independent and works well in current.
We can go to phase2 development of vatz-plugin-cosmoshub

Code Tree

~/vatz-plugin-cosmoshub # tree .
.
├── Makefile
├── plugins
│   ├── cosmos-sdk-blocksync
│   │   ├── go.mod
│   │   ├── go.sum
│   │   ├── main.go
│   │   ├── main_test.go
│   │   ├── Makefile
│   │   ├── mocks
│   │   │   ├── Collector.go
│   │   │   └── Estimator.go
│   │   ├── policy
│   │   │   ├── policy.go
│   │   │   └── policy_test.go
│   │   └── status
│   │       ├── status.go
│   │       └── status_test.go
│   ├── is_alived
│   │   ├── go.mod
│   │   ├── go.sum
│   │   └── main.go
│   └── peer_count
│       ├── go.mod
│       ├── go.sum
│       ├── main.go
│       └── net_info.go
├── README.md
└── rpc
    └── cosmos
        ├── api.go
        ├── api_test.go
        ├── fixtures.go
        ├── go.mod
        ├── go.sum
        └── mocks
            └── Client.go

10 directories, 26 files

heejin-github commented 2 years ago

Phase2 development items

[x] All plugins combine in a module
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/13
[x] refactoring current plugins with new module structure
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/16
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/19
[x] Test code for the plugins
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/15
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/19
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/20
- will be added test code for peer_count
[x] new monitoring plugins
- https://github.com/dsrvlabs/vatz-plugin-cosmoshub/pull/18
[ ] github action for testing : TBD. after open repo to the public.

[ ] Testing on cosmos node
[ ] Find issue during the test
[ ] Freeze for open to the public

xellos00 commented 1 year ago

@heejin-github I've installed following plugins

vatz-plugin-sysutil
vatz-plugin-cosmos-sdk

is there any command that you used pm2 or other method. start & stop plugins easy?

heejin-github commented 1 year ago

@heejin-github I've installed following plugins

vatz-plugin-sysutil

vatz-plugin-cosmos-sdk

is there any command that you used pm2 or other method. start & stop plugins easy?

@xellos00 no longer use pm2. I think It's not familiar way other than node.js. I'm just using a simple start/stop script.

root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# ls
default.yaml  plugin_start.sh  plugin_stop.sh  vatz_start.sh  vatz_stop.sh
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat vatz_start.sh
#!/bin/bash

vatz start --config=/root/dsrv/bin/vatz/default.yaml >> /var/log/vatz/vatz.log 2>&1 &
echo "true"
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat vatz_stop.sh
#!/bin/bash
set -e
set -v

pid=`pidof vatz`
# Kill process
kill -15 $pid

# wait Kill process
sleep 1

while (( `lsof -p $pid | wc -l` > 0 ))
do
        echo `lsof -p $pid | wc -l`
        sleep 1
done

echo "vatz $pid is killed."
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat plugin_start.sh
#!/bin/bash

#cosmos-sdk-blocksync >> /var/log/vatz/cosmos-sdk-blocksync.log 2>&1 &
is_alived --rpcAddr="http://localhost:16657" >> /var/log/vatz/is_alived.log 2>&1 &
peer_count --port=9092 >> /var/log/vatz/peer_count.log 2>&1 &
active_status --port=9097 --rpcURI="http://localhost:11317" --valoperAddr="tpvaloper1cnar5gh3mycnlhw9kkmsg3s792e74xs5x787j6" >> /var/log/vatz/active_status.log 2>&1 &
cpu_monitor >> /var/log/vatz/cpu_monitor.log 2>&1 &
mem_monitor >> /var/log/vatz/mem_monitor.log 2>&1 &
disk_monitor >> /var/log/vatz/disk_monitor.log 2>&1 &
echo "true"
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat plugin_stop.sh
#!/bin/bash

pid=`pidof cosmos-sdk-blocksync`
kill -15 $pid
echo "cosmos-sdk-blocksync $pid is killed."

pid=`pidof is_alived`
kill -15 $pid
echo "is_alived $pid is killed."

pid=`pidof peer_count`
kill -15 $pid
echo "peer_count $pid is killed."

pid=`pidof active_status`
kill -15 $pid
echo "active_status $pid is killed."

pid=`pidof cpu_monitor`
kill -15 $pid
echo "cpu_monitor $pid is killed."

pid=`pidof mem_monitor`
kill -15 $pid
echo "mem_monitor $pid is killed."

pid=`pidof disk_monitor`
kill -15 $pid
echo "disk_monitor $pid is killed."

xellos00 commented 1 year ago

@heejin-github I've installed following plugins

vatz-plugin-sysutil

vatz-plugin-cosmos-sdk

is there any command that you used pm2 or other method. start & stop plugins easy?

@xellos00 no longer use pm2. I think It's not familiar way other than node.js. I'm just using a simple start/stop script.

root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# ls
default.yaml  plugin_start.sh  plugin_stop.sh  vatz_start.sh  vatz_stop.sh
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat vatz_start.sh
#!/bin/bash

vatz start --config=/root/dsrv/bin/vatz/default.yaml >> /var/log/vatz/vatz.log 2>&1 &
echo "true"
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat vatz_stop.sh
#!/bin/bash
set -e
set -v

pid=`pidof vatz`
# Kill process
kill -15 $pid

# wait Kill process
sleep 1

while (( `lsof -p $pid | wc -l` > 0 ))
do
        echo `lsof -p $pid | wc -l`
        sleep 1
done

echo "vatz $pid is killed."
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat plugin_start.sh
#!/bin/bash

#cosmos-sdk-blocksync >> /var/log/vatz/cosmos-sdk-blocksync.log 2>&1 &
is_alived --rpcAddr="http://localhost:16657" >> /var/log/vatz/is_alived.log 2>&1 &
peer_count --port=9092 >> /var/log/vatz/peer_count.log 2>&1 &
active_status --port=9097 --rpcURI="http://localhost:11317" --valoperAddr="tpvaloper1cnar5gh3mycnlhw9kkmsg3s792e74xs5x787j6" >> /var/log/vatz/active_status.log 2>&1 &
cpu_monitor >> /var/log/vatz/cpu_monitor.log 2>&1 &
mem_monitor >> /var/log/vatz/mem_monitor.log 2>&1 &
disk_monitor >> /var/log/vatz/disk_monitor.log 2>&1 &
echo "true"
root@provenance-testnet-validator-dc-seoul:~/dsrv/bin/vatz# cat plugin_stop.sh
#!/bin/bash

pid=`pidof cosmos-sdk-blocksync`
kill -15 $pid
echo "cosmos-sdk-blocksync $pid is killed."

pid=`pidof is_alived`
kill -15 $pid
echo "is_alived $pid is killed."

pid=`pidof peer_count`
kill -15 $pid
echo "peer_count $pid is killed."

pid=`pidof active_status`
kill -15 $pid
echo "active_status $pid is killed."

pid=`pidof cpu_monitor`
kill -15 $pid
echo "cpu_monitor $pid is killed."

pid=`pidof mem_monitor`
kill -15 $pid
echo "mem_monitor $pid is killed."

pid=`pidof disk_monitor`
kill -15 $pid
echo "disk_monitor $pid is killed."

Yeah. so currently there's no options to run those plugins overall in the codes, right?

heejin-github commented 1 year ago

Yeah. so currently there's no options to run those plugins overall in the codes, right?

@xellos00 Yes, you're right. we don't have any option.

heejin-github commented 1 year ago

2022-12-28

Release Candidate : https://github.com/dsrvlabs/vatz-plugin-cosmoshub/commit/ac80cb2ec2c393b3908293604e4087174653a984

heejin-github commented 1 year ago

2022-01-06

v1.0-beta release : https://github.com/dsrvlabs/vatz-plugin-cosmoshub/releases/tag/v1.0-beta

dsrvlabs / vatz