search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.1k stars 834 forks source link

[spark] Introduce reset_consumer procedure #3614

Closed xuzifu666 closed 5 days ago

xuzifu666 commented 6 days ago

Purpose

Introduce reset_consumer procedure in spark

Linked issue: https://github.com/apache/paimon/issues/3615

Tests

API and Format

Documentation

Zouxxyy commented 5 days ago

Just curious, I remember that spark does not support consumer yet

xuzifu666 commented 5 days ago

Just curious, I remember that spark does not support consumer yet

Yes，this procedure not used for spark，it is used for flink paimon job，because some user may need spark to control table manager @Zouxxyy