apache / iotdb

Apache IoTDB
https://iotdb.apache.org/
Apache License 2.0
5.63k stars 1.03k forks source link

Optimize insertRelationalTablet performance #14197

Closed HTHou closed 3 days ago

HTHou commented 4 days ago

Description

This PR is optimizing the insertRelationalTablet performance.

  1. We don't need to checkTTL for each rows of the RelationalTabletNode.
  2. If we can know a RelationalTablet only contains one device, we can optimize the implementation of getIDevices method in RelationalTabletNode.
image

########### Data Mode ########### GROUP_NUMBER=1 IoTDB_TABLE_NUMBER=1 DEVICE_NUMBER=4000 REAL_INSERT_RATE=1.0 SENSOR_NUMBER=50 IS_SENSOR_TS_ALIGNMENT=true IS_OUT_OF_ORDER=false OUT_OF_ORDER_RATIO=0.5 ########### Data Amount ########### OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0:0 SCHEMA_CLIENT_NUMBER=20 DATA_CLIENT_NUMBER=20 LOOP=100 BATCH_SIZE_PER_WRITE=1000 DEVICE_NUM_PER_WRITE=1 START_TIME=2022-01-01T00:00:00+08:00 POINT_STEP=1000 OP_MIN_INTERVAL=0 OP_MIN_INTERVAL_RANDOM=false INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1:1:1:1:1 ENCODINGS=RLE/TS_2DIFF/TS_2DIFF/GORILLA/GORILLA/DICTIONARY/PLAIN/PLAIN/TS_2DIFF/TS_2DIFF COMPRESSOR=LZ4

codecov[bot] commented 4 days ago

Codecov Report

Attention: Patch coverage is 41.17647% with 20 lines in your changes missing coverage. Please review.

Project coverage is 39.63%. Comparing base (d107c2d) to head (9b31d88). Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...er/plan/node/write/RelationalInsertTabletNode.java 36.00% 16 Missing :warning:
...ine/plan/statement/crud/InsertTabletStatement.java 33.33% 2 Missing :warning:
...ngine/plan/relational/planner/RelationPlanner.java 50.00% 1 Missing :warning:
...ryengine/plan/relational/sql/ast/InsertTablet.java 50.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #14197 +/- ## ============================================ - Coverage 39.64% 39.63% -0.01% Complexity 71 71 ============================================ Files 4232 4233 +1 Lines 269870 270000 +130 Branches 32719 32747 +28 ============================================ + Hits 106981 107024 +43 - Misses 162889 162976 +87 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sonarcloud[bot] commented 3 days ago

Quality Gate Passed Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud