Open 7hong opened 1 month ago
Primary key conflicts occur, especially when starting parallelized planning。
processId from:
processId
https://github.com/apache/amoro/blob/4e7fc9b0eb35a6b8768e9ab79a99efed9622b0ea/amoro-ams/src/main/java/org/apache/amoro/server/optimizing/plan/OptimizingPlanner.java#L80-L83
0.7.0
Iceberg
Optimizer
When starting parallel planning, the probability of occurrence is higher
2024-10-14 00:19:45,733 ERROR [plan-executor-thread-499] [org.apache.amoro.server.optimizing.OptimizingQueue] [] - Planning table xx.xx.xx(tableId=30058) failed org.apache.amoro.server.exception.PersistenceException: org.apache.ibatis.exceptions.PersistenceException: ### Error updating database. Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '1728836383424' for key 'PRIMARY' ### The error may exist in org/apache/amoro/server/persistence/mapper/OptimizingMapper.java (best guess) ### The error may involve org.apache.amoro.server.persistence.mapper.OptimizingMapper.insertOptimizingProcess-Inline ### The error occurred while setting parameters ### SQL: INSERT INTO table_optimizing_process(table_id, catalog_name, db_name, table_name ,process_id, target_snapshot_id, target_change_snapshot_id, status, optimizing_type, plan_time, summary, from_sequence, to_sequence) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ### Cause: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry '1728836383424' for key 'PRIMARY'
No response
What happened?
Primary key conflicts occur, especially when starting parallelized planning。
processId
from:https://github.com/apache/amoro/blob/4e7fc9b0eb35a6b8768e9ab79a99efed9622b0ea/amoro-ams/src/main/java/org/apache/amoro/server/optimizing/plan/OptimizingPlanner.java#L80-L83
Affects Versions
0.7.0
What table formats are you seeing the problem on?
Iceberg
What engines are you seeing the problem on?
Optimizer
How to reproduce
When starting parallel planning, the probability of occurrence is higher
Relevant log output
Anything else
No response
Are you willing to submit a PR?
Code of Conduct