[test] Add ut case to check hive clients pool deadlock

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.43k stars 955 forks source link

[test] Add ut case to check hive clients pool deadlock #4493

Closed mxdzs0612 closed 1 week ago

mxdzs0612 commented 1 week ago

Purpose

Linked issue: close #4366

To test if hive client pool deadlock problem has been solved by #4468.

This test case will fail on release-0.9, however passes in master branch.

Tests

org.apache.paimon.hive.HiveCatalogTest#testListTablesLock

API and Format

no.

Documentation

no.

mxdzs0612 commented 1 week ago

It seems it is hard to re-produce this deadlock...

Not that hard, just make 2 threads to call hiveCatalog.listTables at the same time, and assure that there is enough tables in the database, this stuck appears immediately. Just a little hard to construct this test case..

JingsongLi commented 1 week ago

Thanks @mxdzs0612 , +1