tidb-incubator / TiBigData

TiDB connectors for Flink/Hive/Presto
Apache License 2.0
214 stars 56 forks source link

Fix type convert in TiKV sink #224

Closed shiyuhang0 closed 2 years ago

shiyuhang0 commented 2 years ago

What is the purpose of the change

close #223

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

JMH Test

jmh for TiDBRowConverter.toTiRow

jmh for convert object to long

Benchmark                                            (data)    Mode      Cnt        Score   Error  Units
ParseLongBenchMark.convertLong                            1  sample  1232931       26.119 ± 0.899  ns/op
ParseLongBenchMark.convertLong:convertLong·p0.00          1  sample                   ≈ 0          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.50          1  sample                41.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.90          1  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.95          1  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.99          1  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.999         1  sample                84.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.9999        1  sample              6344.544          ns/op
ParseLongBenchMark.convertLong:convertLong·p1.00          1  sample            256256.000          ns/op
ParseLongBenchMark.convertLong                      1000000  sample  1223389       27.422 ± 0.904  ns/op
ParseLongBenchMark.convertLong:convertLong·p0.00    1000000  sample                   ≈ 0          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.50    1000000  sample                41.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.90    1000000  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.95    1000000  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.99    1000000  sample                42.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.999   1000000  sample                84.000          ns/op
ParseLongBenchMark.convertLong:convertLong·p0.9999  1000000  sample              9284.608          ns/op
ParseLongBenchMark.convertLong:convertLong·p1.00    1000000  sample            191232.000          ns/op
ParseLongBenchMark.parseLong                              1  sample  1461371       33.283 ± 3.144  ns/op
ParseLongBenchMark.parseLong:parseLong·p0.00              1  sample                   ≈ 0          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.50              1  sample                41.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.90              1  sample                42.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.95              1  sample                42.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.99              1  sample                42.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.999             1  sample               166.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.9999            1  sample             11241.414          ns/op
ParseLongBenchMark.parseLong:parseLong·p1.00              1  sample            832512.000          ns/op
ParseLongBenchMark.parseLong                        1000000  sample  1204204       68.351 ± 8.961  ns/op
ParseLongBenchMark.parseLong:parseLong·p0.00        1000000  sample                   ≈ 0          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.50        1000000  sample                42.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.90        1000000  sample                83.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.95        1000000  sample                84.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.99        1000000  sample               125.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.999       1000000  sample               375.000          ns/op
ParseLongBenchMark.parseLong:parseLong·p0.9999      1000000  sample             24234.176          ns/op
ParseLongBenchMark.parseLong:parseLong·p1.00        1000000  sample           1861632.000          ns/op

conclusion

shiyuhang0 commented 2 years ago

/run-all-tests

shiyuhang0 commented 2 years ago

/run-all-tests

shiyuhang0 commented 2 years ago

/run-all-tests

shiyuhang0 commented 2 years ago

/run-all-tests

xuanyu66 commented 2 years ago

Since it's a performance-sensitive change. We need to add some benchmarks, maybe use jmh. https://jenkov.com/tutorials/java-performance/jmh.html https://github.com/apache/iceberg/tree/master/spark/v3.2/spark/src/jmh/java/org/apache/iceberg/spark

shiyuhang0 commented 2 years ago

/run-all-tests

xuanyu66 commented 2 years ago

/run-all-tests

shiyuhang0 commented 2 years ago

/run-all-tests

shiyuhang0 commented 2 years ago

/run-all-tests