search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
37.1k stars 5.83k forks source link

Introduce text file analysis tool that detects non-inclusive language in TiDB repo #20439

Open ngaut opened 4 years ago

ngaut commented 4 years ago

woke — Check for insensitive language in your source code https://github.com/get-woke/woke

ghost commented 4 years ago

Here is the output against the main branch:

nullnotnil@ubuntu:~/go/src/github.com/nullnotnil/tidb$ woke
planner/core/planbuilder_test.go:196:0-9: `whiteList` may be insensitive, use `allowlist` instead (warning)
whiteList := []string{"*property.StatsInfo", "*sessionctx.Context", "*mock.Context"}
^
planner/core/planbuilder_test.go:197:99-108: `whiteList` may be insensitive, use `allowlist` instead (warning)
return checkDeepClonedCore(reflect.ValueOf(p1), reflect.ValueOf(p2), typeName(reflect.TypeOf(p1)), whiteList, nil)
                                                                                                   ^
planner/core/planbuilder_test.go:360:0-9: `whiteList` may be insensitive, use `allowlist` instead (warning)
whiteList := []string{"*property.StatsInfo", "*sessionctx.Context", "*mock.Context", "*types.FieldType"}
^
planner/core/planbuilder_test.go:361:101-110: `whiteList` may be insensitive, use `allowlist` instead (warning)
return checkDeepClonedCore(reflect.ValueOf(p), reflect.ValueOf(cloned), typeName(reflect.TypeOf(p)), whiteList, nil)
                                                                                                     ^
planner/core/planbuilder_test.go:367:60-69: `whiteList` may be insensitive, use `allowlist` instead (warning)
func checkDeepClonedCore(v1, v2 reflect.Value, path string, whiteList []string, visited map[visit]bool) error {
                                                            ^
planner/core/planbuilder_test.go:405:88-97: `whiteList` may be insensitive, use `allowlist` instead (warning)
if err := checkDeepClonedCore(v1.Index(i), v2.Index(i), fmt.Sprintf("%v[%v]", path, i), whiteList, visited); err != nil {
                                                                                        ^
planner/core/planbuilder_test.go:426:88-97: `whiteList` may be insensitive, use `allowlist` instead (warning)
if err := checkDeepClonedCore(v1.Index(i), v2.Index(i), fmt.Sprintf("%v[%v]", path, i), whiteList, visited); err != nil {
                                                                                        ^
planner/core/planbuilder_test.go:437:55-64: `whiteList` may be insensitive, use `allowlist` instead (warning)
return checkDeepClonedCore(v1.Elem(), v2.Elem(), path, whiteList, visited)
                                                       ^
planner/core/planbuilder_test.go:445:26-35: `whiteList` may be insensitive, use `allowlist` instead (warning)
for _, whiteName := range whiteList {
                          ^
planner/core/planbuilder_test.go:456:55-64: `whiteList` may be insensitive, use `allowlist` instead (warning)
return checkDeepClonedCore(v1.Elem(), v2.Elem(), path, whiteList, visited)
                                                       ^
planner/core/planbuilder_test.go:459:114-123: `whiteList` may be insensitive, use `allowlist` instead (warning)
if err := checkDeepClonedCore(v1.Field(i), v2.Field(i), fmt.Sprintf("%v.%v", path, typeName(v1.Field(i).Type())), whiteList, visited); err != nil {
                                                                                                                  ^
planner/core/planbuilder_test.go:480:91-100: `whiteList` may be insensitive, use `allowlist` instead (warning)
if err := checkDeepClonedCore(val1, val2, fmt.Sprintf("%v[%v]", path, typeName(k.Type())), whiteList, visited); err != nil {
                                                                                           ^
tidb-server/main_test.go:26:20-25: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// TestRunMain is a dummy test case, which contains only the main function of tidb-server,
                    ^
plugin/plugin_test.go:272:1-10: `whitelist` may be insensitive, use `allowlist` instead (warning)
"whitelist": 1,
 ^
executor/index_advise.go:134:30-35: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// IndexAdviseVarKeyType is a dummy type to avoid naming collision in context.
                              ^
cmd/benchfilesort/main.go:265:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity checks
   ^
cmd/benchfilesort/main.go:295:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity checks
   ^
bindinfo/session_handle.go:112:31-36: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// sessionBindInfoKeyType is a dummy type to avoid naming collision in context.
                               ^
store/tikv/2pc_utils.go:37:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity check for startTS.
   ^
store/tikv/snapshot.go:87:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity check for snapshot version.
   ^
store/tikv/snapshot.go:105:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity check for snapshot version.
   ^
store/tikv/2pc.go:422:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity check for startTS.
   ^
domain/domainctx.go:20:22-27: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// domainKeyType is a dummy type to avoid naming collision in context.
                      ^
store/store_test.go:662:10-15: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
Register("dummy", &brokenStore{})
          ^
store/store_test.go:663:33-38: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
store, err := newStoreWithRetry("dummy://dummy-store", 3)
                                 ^
store/store_test.go:663:41-46: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
store, err := newStoreWithRetry("dummy://dummy-store", 3)
                                         ^
util/filesort/filesort.go:204:3-9: `Sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
// Sanity checks
   ^
util/testleak/fake.go:23:19-24: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// BeforeTest is a dummy implementation when build tag 'leak' is not set.
                   ^
util/testleak/fake.go:27:18-23: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// AfterTest is a dummy implementation when build tag 'leak' is not set.
                  ^
server/server_test.go:897:118-123: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
_, err1 := dbt.db.Exec(`load data local infile '/tmp/load_data_test.csv' into table pn FIELDS TERMINATED BY ',' (c1, @dummy)`)
                                                                                                                      ^
kv/memdb_arena.go:170:3-8: `Dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// Dummy node, so that we can make X.left.up = X.
   ^
session/bootstrap.go:645:13-18: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// This is a dummy upgrade, it checks whether upgradeToVer7 success, if not, do it again.
             ^
session/bootstrap.go:1047:66-75: `blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
// writeDefaultExprPushDownBlacklist writes default expr pushdown blacklist into mysql.expr_pushdown_blacklist
                                                                  ^
infoschema/tables.go:1763:21-26: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// VirtualTable is a dummy table.Table implementation.
                     ^
CHANGELOG.md:60:33-42: `blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
* Refine the usage of expression blacklist (for example, `<` is equivalent to `lt`.) [#11975](https://github.com/pingcap/tidb/pull/11975)
                                 ^
CHANGELOG.md:271:59-68: `Whitelist` may be insensitive, use `allowlist` instead (warning)
* Add the plug-in framework, supporting plugins such as IP Whitelist (Enterprise feature) and Audit Log (Enterprise feature).
                                                           ^
CHANGELOG.md:307:8-17: `blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
* Add a blacklist to prohibit pushing down expressions to Coprocessor
        ^
CHANGELOG.md:387:8-17: `blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
* Add a blacklist to prohibit pushing down expressions to Coprocessor [#10791](https://github.com/pingcap/tidb/pull/10791)
        ^
CHANGELOG.md:554:44-53: `whitelist` may be insensitive, use `allowlist` instead (warning)
* Fix the `ConnectionEvent` error from the `whitelist` plugin that makes TiDB exit [#9889](https://github.com/pingcap/tidb/pull/9889)
                                            ^
CHANGELOG.md:698:8-14: `sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
* Add a sanity check for transactions to avoid false transaction commit [#9559](https://github.com/pingcap/tidb/pull/9559)
        ^
executor/load_data.go:833:27-32: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// loadDataVarKeyType is a dummy type to avoid naming collision in context.
                           ^
executor/opt_rule_blacklist.go:1:1-1: Filename violation: `opt_rule_blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
executor/show.go:1331:82-87: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
e.appendRow([]interface{}{"Replication client", "Server Admin", "To ask where the slave or master servers are"})
                                                                                  ^
executor/show.go:1332:39-44: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
e.appendRow([]interface{}{"Replication slave", "Server Admin", "To read binary log events from the master"})
                                       ^
executor/load_stats.go:42:28-33: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
// loadStatsVarKeyType is a dummy type to avoid naming collision in context.
                            ^
executor/reload_expr_pushdown_blacklist.go:1:1-1: Filename violation: `reload_expr_pushdown_blacklist` may be insensitive, use `denylist`, `blocklist` instead (warning)
executor/executor_test.go:313:50-55: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"Replication client Server Admin To ask where the slave or master servers are",
                                                  ^
executor/executor_test.go:314:13-18: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"Replication slave Server Admin To read binary log events from the master",

And against the docs repo:

keywords.md:541:2-7: `SLAVE` may be insensitive, use `follower`, `replica`, `standby` instead (error)
- SLAVE
  ^
tispark-overview.md:120:13-18: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
./sbin/start-slave.sh spark://spark-master-hostname:7077
             ^
alert-rules.md:1167:61-69: `Blackbox` may be insensitive, use `closed-box` instead (error)
* View the ping latency between the two nodes on the Grafana Blackbox Exporter dashboard to check whether it is too high.
                                                             ^
sync-diff-inspector/upstream-downstream-diff.md:58:54-59: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
> - In some versions of TiDB Binlog, `master-ts` and `slave-ts` are stored in `ts-map`. `master-ts` is equivalent to `primary-ts` and `slave-ts` is equivalent to `secondary-ts`.
                                                      ^
sync-diff-inspector/upstream-downstream-diff.md:58:135-140: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
> - In some versions of TiDB Binlog, `master-ts` and `slave-ts` are stored in `ts-map`. `master-ts` is equivalent to `primary-ts` and `slave-ts` is equivalent to `secondary-ts`.
                                                                                                                                       ^
tidb-lightning/deploy-tidb-lightning.md:103:0-5: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
dummy:
^
tidb-lightning/tidb-lightning-backends.md:338:0-5: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
dummy:
^
tidb-lightning/tidb-lightning-backends.md:351:0-5: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
dummy:
^
releases/release-4.0.3.md:99:19-29: `white-list` may be insensitive, use `allowlist` instead (warning)
- Deprecate `black-white-list` with a newer and easier-to-understand filter format [#332](https://github.com/pingcap/tidb-lightning/pull/332)
                   ^
releases/release-2.1.6.md:20:8-14: `sanity` may be insensitive, use `confidence`, `quick check`, `coherence check` instead (error)
- Add a sanity check for transactions to avoid false transaction commit [#9559](https://github.com/pingcap/tidb/pull/9559)
        ^
sql-statements/sql-statement-show-privileges.md:45:85-90: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
| Replication client      | Server Admin                          | To ask where the slave or master servers are          |
                                                                                     ^
sql-statements/sql-statement-show-privileges.md:46:14-19: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
| Replication slave       | Server Admin                          | To read binary log events from the master             |
              ^
tidb-binlog/binlog-consumer-client.md:4:28-33: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
aliases: ['/tidb/dev/binlog-slave-client','/docs/dev/tidb-binlog/binlog-slave-client/','/docs/dev/reference/tidb-binlog/binlog-slave-client/']
                            ^
tidb-binlog/binlog-consumer-client.md:4:72-77: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
aliases: ['/tidb/dev/binlog-slave-client','/docs/dev/tidb-binlog/binlog-slave-client/','/docs/dev/reference/tidb-binlog/binlog-slave-client/']
                                                                        ^
tidb-binlog/binlog-consumer-client.md:4:127-132: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
aliases: ['/tidb/dev/binlog-slave-client','/docs/dev/tidb-binlog/binlog-slave-client/','/docs/dev/reference/tidb-binlog/binlog-slave-client/']
                                                                                                                               ^
user-account-management.md:125:13-18: `dummy` may be insensitive, use `placeholder`, `sample` instead (error)
CREATE USER 'dummy'@'localhost';
             ^
syncer-overview.md:81:17-22: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
to specify MySQL slave sever-id (default 101)
                 ^
syncer-overview.md:206:57-62: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
2016/10/27 15:22:01 binlogsyncer.go:130: [info] register slave for master server 127.0.0.1:3306
                                                         ^
syncer-overview.md:431:20-25: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
select, replication slave, replication client
                    ^
etc/Drainer.json:286:10-15: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"title": "slave upper boundary",
          ^
etc/Drainer.json:364:10-15: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"title": "slave  lower boundary",
          ^
etc/Drainer.json:442:10-15: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"title": "slave position",
          ^
etc/Drainer.json:597:10-15: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
"title": "slave tikv query",
          ^
releases/release-3.0.17.md:29:10-20: `white-list` may be insensitive, use `allowlist` instead (warning)
- `[black-white-list]` has been deprecated with a newer, easier-to-understand filter format [#332](https://github.com/pingcap/tidb-lightning/pull/332)
          ^
tidb-troubleshooting-map.md:486:87-92: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
tidb-troubleshooting-map.md:486:220-225: `slave` may be insensitive, use `follower`, `replica`, `standby` instead (error)
releases/release-3.0.0-rc.1.md:45:44-53: `whitelist` may be insensitive, use `allowlist` instead (warning)
- Fix the `ConnectionEvent` error from the `whitelist` plugin that makes TiDB exit [#9889](https://github.com/pingcap/tidb/pull/9889)
                                            ^
releases/release-3.0-ga.md:31:59-68: `Whitelist` may be insensitive, use `allowlist` instead (warning)
- Add the plug-in framework, supporting plugins such as IP Whitelist (**Enterprise**) and Audit Log (**Enterprise**).
jjshanks commented 4 years ago

@nullnotnil / @ngaut Would a good approach for this be to add a github action for woke?

Do you want to add an ignore file for existing usage or also change them?

ghost commented 4 years ago

@jjshanks yes, but we can't currently use GitHub actions on this repo :(

In the local development environment make dev runs a set of checks. I think this would be a good place to check all files that are being modified.

valkyr13 commented 1 year ago

can i work on this @jjshanks

jjshanks commented 1 year ago

@valkyr13 I am not currently working on it. Go for it :)

leungyukshing commented 1 year ago

/assign

leungyukshing commented 1 year ago

@ngaut Hi, do we still want to add the check into make dev?