search

pingcap / tiup

A component manager for TiDB
https://tiup.io
Apache License 2.0
417 stars 310 forks source link

not cache the output of `ss -tln` command when checking wether components are started/stopped #2298

Closed KanShiori closed 11 months ago

KanShiori commented 11 months ago

What problem does this PR solve?

Close #2267

What is changed and how it works?

Add a function UnwarpCheckPointExecutor to get a real executor without checkpoints.

Use UnwarpCheckPointExecutor to executor ss -tln when checking wether components are started/stopped.

Check List

Tests

  1. Deploy a new cluster and check audit logs.

    The ss -tln command isn't cached when starting components.

    // ...
2023-11-01T02:34:40.179Z        INFO    Starting component blackbox_exporter
2023-11-01T02:34:40.179Z        INFO            Starting instance 127.0.0.1
2023-11-01T02:34:41.152Z        INFO    LocalCommand    {"cmd": "export LANG=C; PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin /usr/bin/sudo -H -u root bash -c \"cd; systemctl daemon-reload && systemctl start blackbox_exporter-9115.service\"", "stdout": "", "stderr": ""}
2023-11-01T02:34:41.153Z        INFO    CheckPoint      {"host": "127.0.0.1", "port": 22, "user": "tidb", "sudo": true, "cmd": "systemctl daemon-reload && systemctl start blackbox_exporter-9115.service", "stdout": "", "stderr": "", "__hash__": "7634a3f7", "__func__": "github.com/pingcap/tiup/pkg/cluster/executor.(*CheckPointExecutor).Execute", "hit": false}
2023-11-01T02:34:41.189Z        INFO    LocalCommand    {"cmd": "export LANG=C; PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin /usr/bin/sudo -H -u tidb bash -c \"cd; ss -ltn\"", "stdout": "State  Recv-Q Send-Q      Local Address:Port  Peer Address:PortProcess\nLISTEN 0      70              127.0.0.1:33060      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:45347      0.0.0.0:*          \nLISTEN 0      200             127.0.0.1:8234       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:9090       0.0.0.0:*          \nLISTEN 0      511             127.0.0.1:42251      0.0.0.0:*          \nLISTEN 0      4096           127.0.0.54:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2380       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2379       0.0.0.0:*          \nLISTEN 0      4096        127.0.0.53%lo:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:3000       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44301      0.0.0.0:*          \nLISTEN 0      151             127.0.0.1:3306       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44037      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:4000       0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20292      0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20180      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:12020      0.0.0.0:*          \nLISTEN 0      4096                    *:9100             *:*          \nLISTEN 0      4096                    *:22               *:*          \nLISTEN 0      4096                    *:8300             *:*          \nLISTEN 0      4096                    *:10080            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096                    *:9527             *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160
*:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \n", "stderr": ""}
2023-11-01T02:34:42.229Z        INFO    LocalCommand    {"cmd": "export LANG=C; PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin /usr/bin/sudo -H -u tidb bash -c \"cd; ss -ltn\"", "stdout": "State  Recv-Q Send-Q      Local Address:Port  Peer Address:PortProcess\nLISTEN 0      70              127.0.0.1:33060      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:45347      0.0.0.0:*          \nLISTEN 0      200             127.0.0.1:8234       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:9090       0.0.0.0:*          \nLISTEN 0      511             127.0.0.1:42251      0.0.0.0:*          \nLISTEN 0      4096           127.0.0.54:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2380       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2379       0.0.0.0:*          \nLISTEN 0      4096        127.0.0.53%lo:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:3000       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44301      0.0.0.0:*          \nLISTEN 0      151             127.0.0.1:3306       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44037      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:4000       0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20292      0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20180      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:12020      0.0.0.0:*          \nLISTEN 0      4096                    *:9100             *:*          \nLISTEN 0      4096                    *:9115             *:*          \nLISTEN 0      4096                    *:22               *:*          \nLISTEN 0      4096                    *:8300             *:*          \nLISTEN 0      4096                    *:10080            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096                    *:9527
*:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \n", "stderr": ""}
2023-11-01T02:34:42.229Z        INFO            Start 127.0.0.1 success
// ...

  2. Upgrade the cluster and check audit logs.

    The ss -tln command isn't cached when stopping components.

    2023-11-01T02:36:29.240Z        INFO    Stopping component node_exporter
2023-11-01T02:36:29.246Z        INFO            Stopping instance 127.0.0.1
2023-11-01T02:36:30.044Z        INFO    LocalCommand    {"cmd": "export LANG=C; PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin /usr/bin/sudo -H -u root bash -c \"cd; systemctl daemon-reload && systemctl stop node_exporter-9100.service\"", "stdout": "", "stderr": ""}
2023-11-01T02:36:30.044Z        INFO    CheckPoint      {"host": "127.0.0.1", "port": 22, "user": "tidb", "sudo": true, "cmd": "systemctl daemon-reload && systemctl stop node_exporter-9100.service", "stdout": "", "stderr": "", "__hash__": "7634a3f7", "__func__": "github.com/pingcap/tiup/pkg/cluster/executor.(*CheckPointExecutor).Execute", "hit": false}
2023-11-01T02:36:30.072Z        INFO    LocalCommand    {"cmd": "export LANG=C; PATH=$PATH:/bin:/sbin:/usr/bin:/usr/sbin /usr/bin/sudo -H -u tidb bash -c \"cd; ss -ltn\"", "stdout": "State  Recv-Q Send-Q      Local Address:Port  Peer Address:PortProcess\nLISTEN 0      70              127.0.0.1:33060      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:45347      0.0.0.0:*          \nLISTEN 0      200             127.0.0.1:8234       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:9090       0.0.0.0:*          \nLISTEN 0      511             127.0.0.1:42251      0.0.0.0:*          \nLISTEN 0      4096           127.0.0.54:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2380       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:2379       0.0.0.0:*          \nLISTEN 0      4096        127.0.0.53%lo:53         0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:3000       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44301      0.0.0.0:*          \nLISTEN 0      151             127.0.0.1:3306       0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:44037      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:4000       0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20292      0.0.0.0:*          \nLISTEN 0      128             127.0.0.1:20180      0.0.0.0:*          \nLISTEN 0      4096            127.0.0.1:12020      0.0.0.0:*          \nLISTEN 0      4096                    *:9115             *:*          \nLISTEN 0      4096                    *:22               *:*          \nLISTEN 0      4096                    *:8300             *:*          \nLISTEN 0      4096                    *:10080            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:3930             *:*          \nLISTEN 0      4096                    *:9527             *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160
*:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20160            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \nLISTEN 0      4096   [::ffff:127.0.0.1]:20170            *:*          \n", "stderr": ""}
2023-11-01T02:36:30.072Z        INFO            Stop 127.0.0.1 success
    • No code

Code changes

Side effects

Related changes

Release notes:

NONE
ti-chi-bot[bot] commented 11 months ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign srstack for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/pingcap/tiup/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
codecov-commenter commented 11 months ago

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (1477739) 55.01% compared to head (9897aa7) 55.33%.

:exclamation: Current head 9897aa7 differs from pull request most recent head 3c878fb. Consider uploading reports for the commit 3c878fb to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #2298 +/- ## ========================================== + Coverage 55.01% 55.33% +0.32% ========================================== Files 326 326 Lines 34473 34479 +6 ========================================== + Hits 18964 19076 +112 + Misses 13237 13117 -120 - Partials 2272 2286 +14 ``` | [Flag](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [cluster](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `44.47% <71.43%> (+0.39%)` | :arrow_up: | | [dm](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `25.47% <71.43%> (+0.01%)` | :arrow_up: | | [playground](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `15.43% <0.00%> (-<0.01%)` | :arrow_down: | | [tiup](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `15.69% <0.00%> (+0.02%)` | :arrow_up: | | [unittest](https://app.codecov.io/gh/pingcap/tiup/pull/2298/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | `22.43% <0.00%> (-<0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/pingcap/tiup/pull/2298?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap) | Coverage Δ | | |---|---|---| | [pkg/cluster/api/tidbapi.go](https://app.codecov.io/gh/pingcap/tiup/pull/2298?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#diff-cGtnL2NsdXN0ZXIvYXBpL3RpZGJhcGkuZ28=) | `0.00% <ø> (ø)` | | | [pkg/cluster/module/wait\_for.go](https://app.codecov.io/gh/pingcap/tiup/pull/2298?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#diff-cGtnL2NsdXN0ZXIvbW9kdWxlL3dhaXRfZm9yLmdv) | `80.95% <100.00%> (ø)` | | | [pkg/cluster/executor/executor.go](https://app.codecov.io/gh/pingcap/tiup/pull/2298?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap#diff-cGtnL2NsdXN0ZXIvZXhlY3V0b3IvZXhlY3V0b3IuZ28=) | `64.00% <66.67%> (+0.17%)` | :arrow_up: | ... and [10 files with indirect coverage changes](https://app.codecov.io/gh/pingcap/tiup/pull/2298/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pingcap)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ti-chi-bot[bot] commented 11 months ago

[LGTM Timeline notifier]

Timeline: