search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.47k stars 969 forks source link

[hive] HiveMigrator support set parallelism for procedures #4177

Closed xuzifu666 closed 2 months ago

xuzifu666 commented 2 months ago

Purpose

Linked issue: close #xxx Currently migrate table/file/database cannot set thread parallelism, user can not overclock cpu to improve migrate speed or reduce cpu cost for avoid impacting other process in machine , this pr is support parallelism change.

Tests

API and Format

Documentation

JingsongLi commented 2 months ago

Hi @xuzifu666 , parallelism should be a int.

xuzifu666 commented 2 months ago

Hi @xuzifu666 , parallelism should be a int.

Thanks, had change parameter parallelism from String to Integer. @JingsongLi