search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.13k stars 842 forks source link

[spark]Support get full table name include database #3553

Closed waywtdcc closed 2 days ago

waywtdcc commented 2 weeks ago

Purpose

Support get full table name include database ranger spark auth plugin check auth error

image

Tests

API and Format

Documentation

Zouxxyy commented 2 weeks ago

This is the define of String name()

  /**
   * A name to identify this table. Implementations should provide a meaningful name, like the
   * database and table name from catalog, or the location of files for this table.
   */
  String name();

I don't think it is a good idea for range to get database through it, because this interface is just a name and users may define it arbitrarily.

However, enriching this name helps display information in spark UI, I remember iceberg is catalog.db.name now.

Zouxxyy commented 2 weeks ago

This should be fixed in kyuubi range plugin

waywtdcc commented 2 weeks ago

But it's not unique if there is only one table name, which doesn't feel good.

Zouxxyy commented 2 weeks ago

But it's not unique if there is only one table name, which doesn't feel good.

Yes, as I said before, this name will eventually be displayed in the UI, it'll be great to like catalog.db.name

The current implementation requires modifying the constructor, it would be nice if the paimonTable could include it.

Zouxxyy commented 2 days ago

https://github.com/apache/paimon/pull/3680 focus on the same problem and merged... sorry for that

JingsongLi commented 2 days ago

Fixed in: https://github.com/apache/paimon/pull/3680 https://github.com/apache/paimon/pull/3686