Storage and computing node installation script update

[jd-zhang]

Issue migrated from trac ticket # 212 www.kunlunbase.com

component: kunlun-storage | priority: major

2021-11-05 14:26:51: zhaowei@zettadb.com created the issue

---

= Json parameters

All below needed parameters should be (and have been) added to json config files for comp and storage nodes already.

= Storage node new config params

== install_mysql.py and its json config file Must set the shard_id and cluster_id specified in cluster config json file to the .cnf file of each of the shard's storage nodes, the var is 'shard_id' and 'cluster_id' in storage nodes. Currently the var exists but it's not set with valid id and not stored in config files.

== add_shards.py and its json config file the json config file contains these new fields:

1. Mandatory fields Require 'ro_weight', master_priority and svr_node_id fields for each storage node in the json config file when installing a kunlun cluster. Store the 3 field of each storage node to metadata shard's shard_nodes table, and store ro_weight and svr_node_id into each computing node's pg_shard_node table.

• ro_weight The ro_weight a non-negative integer, if non zero, the bigger the higher chance to be used for slave read; if 0, never used for slave reads. it's OK for all nodes have 0.

• master_priority: non-negative integer, the bigger the higher chance to be elected a master, set to 0 if never to be elected as master.

• svr_node_id which computer server the node is installed in, it's a FK to server_nodes.

2. resource limit fields Optionally allow resource limit fields including: cpu_cores initial_storage_GB max_storage_GB innodb_buffer_pool_MB rocksdb_buffer_pool_MB fields of shard_nodes table, for each storage node in the json config file. these will be useful for cloud DBaaS.

3. Extra fields

   • channel_name stored as a json string, and the json string is to be stored as a json string into shard_nodes.extra_props. this is the channel name of the 'FOR CHANNEL XXX' clause of the 'CHANGE MASTER TO ... FOR CHANNEL ' when doing master election.

Store all above fields into shard_nodes table.

= Computing node new config params

For add_comp_node.py and its json config file, below new fields should be added into the json config file and stored into comp_nodes table by add_comp_node.py

1. Require svr_node_id field for each computing node in the json config file when installing a kunlun cluster. Store the field into comp_nodes table of metadata shard.

2. Optionally allow resource limit fields including: cpu_cores max_mem, max_conns fields of comp_nodes table, for each computing node in the json config file. these will be useful for cloud DBaaS.

Store such fields into comp_nodes table of metadata shard.

[jd-zhang]

2021-12-22 10:01:30: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-22 10:01:30: zhaowei@zettadb.com changed title from Storage node installation script update to Storage and computing node installation script update

[jd-zhang]

2021-12-22 10:14:02: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-22 10:14:31: zhaowei@zettadb.com changed type from defect to task

[jd-zhang]

2021-12-29 11:14:47: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-29 12:03:27: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-29 12:22:20: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-29 15:40:27: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2021-12-29 15:46:07: zhaowei@zettadb.com edited the issue description

[jd-zhang]

2022-03-28 14:31:18: zhangjindong@zettadb.com