search

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.32k stars 2.42k forks source link

Single table multiple write conflicts！ #10333

Open 13083718081 opened 9 months ago

13083718081 commented 9 months ago

How to correctly configure the single-table multi-write mode. Currently, configured according to the official website's method, the task keeps restarting when generating a compact plan.

Concurrency control configuration in Flink jobs： 'hoodie.write.concurrency.mode'='optimistic_concurrency_control', 'hoodie.cleaner.policy.failed.writes'='LAZY', 'hoodie.write.lock.provider'='org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider', 'hoodie.write.lock.zookeeper.url'='192.168.1.10', 'hoodie.write.lock.zookeeper.port'='2181', 'hoodie.write.lock.zookeeper.lock_key'='user_test_1', 'hoodie.write.lock.zookeeper.base_path'='/test'

Process Description: Writing data into different partitions is fine, but when the number of compaction.delta_commits reaches the set value, an error is reported when generating the compaction plan.

Error message: Caused by: org.apache.hudi.exception.HoodieWriteConflictException: java.util.ConcurrentModificationException: Cannot resolve conflicts for overlapping writes at org.apache.hudi.client.transaction.SimpleConcurrentFileWritesConflictResolutionStrategy.resolveConflict(SimpleConcurrentFileWritesConflictResolutionStrategy.java:111) at org.apache.hudi.client.utils.TransactionUtils.lambda$resolveWriteConflictIfAny$0(TransactionUtils.java:89) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) at org.apache.hudi.client.utils.TransactionUtils.resolveWriteConflictIfAny(TransactionUtils.java:83) at org.apache.hudi.client.BaseHoodieClient.resolveWriteConflict(BaseHoodieClient.java:202) at org.apache.hudi.client.BaseHoodieWriteClient.preCommit(BaseHoodieWriteClient.java:346) at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:232) at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:112) at org.apache.hudi.client.HoodieFlinkWriteClient.commit(HoodieFlinkWriteClient.java:75) at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:201) at org.apache.hudi.sink.StreamWriteOperatorCoordinator.doCommit(StreamWriteOperatorCoordinator.java:564) at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:540) at org.apache.hudi.sink.StreamWriteOperatorCoordinator.commitInstant(StreamWriteOperatorCoordinator.java:509) at org.apache.hudi.sink.StreamWriteOperatorCoordinator.lambda$initInstant$6(StreamWriteOperatorCoordinator.java:419) at org.apache.hudi.sink.utils.NonThrownExecutor.lambda$wrapAction$0(NonThrownExecutor.java:130) ... 3 more

Please take a look to see where the problem is, thanks！

danny0405 commented 8 months ago

You can have a try of our new NB-CC concurrency mode, here is what it looks like with Flink options:

-- NB-CC demo

CREATE TABLE sourceT (
  uuid varchar(20),
  name varchar(10),
  age int,
  ts timestamp(3),
  `partition` as 'par1'
) WITH (
  'connector' = 'datagen',
  'rows-per-second' = '200'
);

create table t1(
  uuid varchar(20),
  name varchar(10),
  age int,
  ts timestamp(3),
  `partition` varchar(20)
)
with (
  'connector' = 'hudi',
  'path' = '/Users/chenyuzhao/workspace/hudi-demo/t1',
  'table.type' = 'MERGE_ON_READ',
  'index.type' = 'BUCKET',
  'hoodie.write.concurrency.mode' = 'NON_BLOCKING_CONCURRENCY_CONTROL',
  'write.tasks' = '2'
);

create table t1_2(
  uuid varchar(20),
  name varchar(10),
  age int,
  ts timestamp(3),
  `partition` varchar(20)
)
with (
  'connector' = 'hudi',
  'path' = '/Users/chenyuzhao/workspace/hudi-demo/t1',
  'table.type' = 'MERGE_ON_READ',
  'index.type' = 'BUCKET',
  'hoodie.write.concurrency.mode' = 'NON_BLOCKING_CONCURRENCY_CONTROL',
  'write.tasks' = '2',
  'compaction.schedule.enabled' = 'false',
  'compaction.async.enabled' = 'false',
  'clean.async.enabled' = 'false'
);

insert into t1 select * from sourceT;
insert into t1_2 select * from sourceT;
13083718081 commented 8 months ago

After adjusting the configuration, concurrent conflicts can be resolved, but it impacts compaction. Initially, all is well, but with time, data will only be written to the .log file, and parquet files will cease to be generated.Why is this the case?

ad1happy2go commented 8 months ago

@13083718081 So with NON_BLOCKING_CONCURRENCY_CONTROL, you saying the compaction stops working after some time? Do you see compaction instant in timeline?

13083718081 commented 8 months ago

Yes, compaction will no longer be carried out after a period of time, and this situation will arise. 2023-12-19 14:39 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219143955835.log.1_0-1-0 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144011932.log.1_0-1-0 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/00000001-0000-0000-0000-000000000000_0-1-0_20231219144030002.parquet 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/00000001-0000-0000-0000-000000000000_0-1-0_20231219143957434.parquet 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144120853.log.1_0-1-0 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144127159.log.1_0-1-0 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144131181.log.1_0-1-0 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144135288.log.1_0-1-0 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/00000001-0000-0000-0000-000000000000_0-1-0_20231219144130206.parquet 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/00000001-0000-0000-0000-000000000000_0-1-0_20231219144149636.parquet 2023-12-19 14:43 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144347324.log.1_0-1-0 2023-12-19 14:43 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144354327.log.1_0-1-0 2023-12-19 14:44 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144401978.log.1_0-1-0 2023-12-19 14:44 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144411632.log.1_0-1-0 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144710614.log.1_0-1-0 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144721807.log.1_0-1-0 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144743281.log.1_0-1-0 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219144747000.log.1_0-1-0 2023-12-19 14:51 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145105289.log.1_0-1-0 2023-12-19 14:51 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145108864.log.1_0-1-0 2023-12-19 14:54 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145421315.log.1_0-1-0 2023-12-19 14:54 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145439876.log.1_0-1-0 2023-12-19 14:56 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145658832.log.1_0-1-0 2023-12-19 14:57 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145709030.log.1_0-1-0 2023-12-19 14:57 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145737111.log.1_0-1-0 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219145752796.log.1_0-1-0 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150115205.log.1_0-1-0 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150126043.log.1_0-1-0 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150152287.log.1_0-1-0 2023-12-19 15:02 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150202683.log.1_0-1-0 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150309403.log.1_0-1-0 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150315781.log.1_0-1-0 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150529639.log.1_0-1-0 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150537188.log.1_0-1-0 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150542398.log.1_0-1-0 2023-12-19 15:08 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150821480.log.1_0-1-0 2023-12-19 15:09 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150903962.log.1_0-1-0 2023-12-19 15:09 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150941230.log.1_0-1-0 2023-12-19 15:09 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219150947614.log.1_0-1-0 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151201787.log.1_0-1-0 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151208099.log.1_0-1-0 2023-12-19 15:13 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151336197.log.1_0-1-0 2023-12-19 15:13 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151340094.log.1_0-1-0 2023-12-19 15:16 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151659048.log.1_0-1-0 2023-12-19 15:17 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219151701871.log.1_0-1-0 2023-12-19 15:25 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152500335.log.1_0-1-0 2023-12-19 15:25 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152508990.log.1_0-1-0 2023-12-19 15:25 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152517941.log.1_0-1-0 2023-12-19 15:26 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152657096.log.1_0-1-0 2023-12-19 15:27 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152710776.log.1_0-1-0 2023-12-19 15:27 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152706908.log.1_0-1-0 2023-12-19 15:29 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152904681.log.1_0-1-0 2023-12-19 15:29 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219152909111.log.1_0-1-0 2023-12-19 15:30 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219153004216.log.1_0-1-0 2023-12-19 15:30 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219153008318.log.1_0-1-0 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219153112921.log.1_0-1-0 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/202311/30/.00000001-0000-0000-0000-000000000000_20231219153129415.log.1_0-1-0

compaction instant in timeline： 2023-12-19 14:39 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219143912267.compaction.requested 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219143957434.compaction.inflight 2023-12-19 14:39 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219143957434.compaction.requested 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144018523.compaction.inflight 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144018523.compaction.requested 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144030002.compaction.inflight 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144030002.compaction.requested 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144037735.compaction.inflight 2023-12-19 14:40 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144037735.compaction.requested 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144111809.compaction.inflight 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144111809.compaction.requested 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144130206.compaction.inflight 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144130206.compaction.requested 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144149636.compaction.inflight 2023-12-19 14:41 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144149636.compaction.requested 2023-12-19 14:43 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144331537.compaction.inflight 2023-12-19 14:43 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144331537.compaction.requested 2023-12-19 15:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144353311.compaction.inflight 2023-12-19 14:43 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144353311.compaction.requested 2023-12-19 14:44 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144444698.compaction.inflight 2023-12-19 14:44 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144444698.compaction.requested 2023-12-19 14:45 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144458533.compaction.inflight 2023-12-19 14:44 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144458533.compaction.requested 2023-12-19 14:45 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144524609.compaction.inflight 2023-12-19 14:45 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144524609.compaction.requested 2023-12-19 14:45 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144547474.compaction.inflight 2023-12-19 14:45 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144547474.compaction.requested 2023-12-19 14:46 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144615520.compaction.inflight 2023-12-19 14:46 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144615520.compaction.requested 2023-12-19 14:46 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144642277.compaction.inflight 2023-12-19 14:46 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144642277.compaction.requested 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144728448.compaction.inflight 2023-12-19 14:47 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144728448.compaction.requested 2023-12-19 14:48 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144803823.compaction.inflight 2023-12-19 14:48 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144803823.compaction.requested 2023-12-19 14:49 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144857221.compaction.inflight 2023-12-19 14:48 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144857221.compaction.requested 2023-12-19 14:49 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144937992.compaction.inflight 2023-12-19 14:49 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219144937992.compaction.requested 2023-12-19 14:51 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145125768.compaction.inflight 2023-12-19 14:51 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145125768.compaction.requested 2023-12-19 14:52 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145213145.compaction.inflight 2023-12-19 14:52 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145213145.compaction.requested 2023-12-19 14:52 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145248745.compaction.inflight 2023-12-19 14:52 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145248745.compaction.requested 2023-12-19 14:53 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145317399.compaction.inflight 2023-12-19 14:53 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145317399.compaction.requested 2023-12-19 14:54 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145437973.compaction.inflight 2023-12-19 14:54 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145437973.compaction.requested 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145506874.compaction.inflight 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145506874.compaction.requested 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145522536.compaction.inflight 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145522536.compaction.requested 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145549406.compaction.inflight 2023-12-19 14:55 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145549406.compaction.requested 2023-12-19 14:56 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145626036.compaction.inflight 2023-12-19 14:56 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145626036.compaction.requested 2023-12-19 14:56 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145643718.compaction.inflight 2023-12-19 14:56 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145643718.compaction.requested 2023-12-19 14:57 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145731309.compaction.inflight 2023-12-19 14:57 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145731309.compaction.requested 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145806337.compaction.inflight 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145806337.compaction.requested 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145816719.compaction.inflight 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145816719.compaction.requested 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145846868.compaction.inflight 2023-12-19 14:58 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145846868.compaction.requested 2023-12-19 14:59 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145924436.compaction.inflight 2023-12-19 14:59 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145924436.compaction.requested 2023-12-19 14:59 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145946741.compaction.inflight 2023-12-19 14:59 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219145946741.compaction.requested 2023-12-19 15:00 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150028787.compaction.inflight 2023-12-19 15:00 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150028787.compaction.requested 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150102593.compaction.inflight 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150102593.compaction.requested 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150119751.compaction.inflight 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150119751.compaction.requested 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150131511.compaction.inflight 2023-12-19 15:01 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150131511.compaction.requested 2023-12-19 15:02 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150214709.compaction.inflight 2023-12-19 15:02 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150214709.compaction.requested 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150307435.compaction.inflight 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150307435.compaction.requested 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150321728.compaction.inflight 2023-12-19 15:03 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150321728.compaction.requested 2023-12-19 15:04 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150358984.compaction.inflight 2023-12-19 15:04 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150358984.compaction.requested 2023-12-19 15:04 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150449437.compaction.inflight 2023-12-19 15:04 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150449437.compaction.requested 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150508781.compaction.inflight 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150508781.compaction.requested 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150541168.compaction.inflight 2023-12-19 15:05 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150541168.compaction.requested 2023-12-19 15:06 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150611256.compaction.inflight 2023-12-19 15:06 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150611256.compaction.requested 2023-12-19 15:06 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150639672.compaction.inflight 2023-12-19 15:06 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150639672.compaction.requested 2023-12-19 15:07 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150700481.compaction.inflight 2023-12-19 15:07 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150700481.compaction.requested 2023-12-19 15:07 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150727986.compaction.inflight 2023-12-19 15:07 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150727986.compaction.requested 2023-12-19 15:08 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150803208.compaction.inflight 2023-12-19 15:08 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150803208.compaction.requested 2023-12-19 15:08 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150840113.compaction.inflight 2023-12-19 15:08 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150840113.compaction.requested 2023-12-19 15:09 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150928861.compaction.inflight 2023-12-19 15:09 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219150928861.compaction.requested 2023-12-19 15:10 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151024880.compaction.inflight 2023-12-19 15:10 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151024880.compaction.requested 2023-12-19 15:10 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151048921.compaction.inflight 2023-12-19 15:10 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151048921.compaction.requested 2023-12-19 15:11 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151114061.compaction.inflight 2023-12-19 15:11 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151114061.compaction.requested 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151151862.compaction.inflight 2023-12-19 15:11 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151151862.compaction.requested 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151207018.compaction.inflight 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151207018.compaction.requested 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151243853.compaction.inflight 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151243853.compaction.requested 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151251521.compaction.inflight 2023-12-19 15:12 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151251521.compaction.requested 2023-12-19 15:13 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151350645.compaction.inflight 2023-12-19 15:13 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151350645.compaction.requested 2023-12-19 15:14 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151455381.compaction.inflight 2023-12-19 15:14 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151455381.compaction.requested 2023-12-19 15:16 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151630622.compaction.inflight 2023-12-19 15:16 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151630622.compaction.requested 2023-12-19 15:17 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151752366.compaction.inflight 2023-12-19 15:17 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151752366.compaction.requested 2023-12-19 15:18 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151821652.compaction.inflight 2023-12-19 15:18 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151821652.compaction.requested 2023-12-19 15:19 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151903188.compaction.inflight 2023-12-19 15:19 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151903188.compaction.requested 2023-12-19 15:19 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151927627.compaction.inflight 2023-12-19 15:19 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151927627.compaction.requested 2023-12-19 15:20 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151955070.compaction.inflight 2023-12-19 15:19 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219151955070.compaction.requested 2023-12-19 15:20 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152024067.compaction.inflight 2023-12-19 15:20 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152024067.compaction.requested 2023-12-19 15:21 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152101408.compaction.inflight 2023-12-19 15:21 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152101408.compaction.requested 2023-12-19 15:21 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152129646.compaction.inflight 2023-12-19 15:21 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152129646.compaction.requested 2023-12-19 15:22 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152150498.compaction.inflight 2023-12-19 15:21 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152150498.compaction.requested 2023-12-19 15:22 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152216884.compaction.inflight 2023-12-19 15:22 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152216884.compaction.requested 2023-12-19 15:22 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152241279.compaction.inflight 2023-12-19 15:22 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152241279.compaction.requested 2023-12-19 15:23 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152312677.compaction.inflight 2023-12-19 15:23 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152312677.compaction.requested 2023-12-19 15:23 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152341079.compaction.inflight 2023-12-19 15:23 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152341079.compaction.requested 2023-12-19 15:24 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152417693.compaction.inflight 2023-12-19 15:24 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152417693.compaction.requested 2023-12-19 15:29 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152939134.compaction.inflight 2023-12-19 15:29 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219152939134.compaction.requested 2023-12-19 15:30 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153002303.compaction.inflight 2023-12-19 15:30 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153002303.compaction.requested 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153102389.compaction.inflight 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153102389.compaction.requested 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153144240.compaction.inflight 2023-12-19 15:31 hdfs://192.168.1.10:9000/user/test/user_test_1/.hoodie/20231219153144240.compaction.requested

ad1happy2go commented 8 months ago

cc @danny0405