ExpediaGroup / waggle-dance

Hive federation service. Enables disparate tables to be concurrently accessed across multiple Hive deployments.
Apache License 2.0
261 stars 71 forks source link

The current rate limiting is not effective in non KRB environments #323

Open flaming-archer opened 3 days ago

flaming-archer commented 3 days ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. For example - I'm always frustrated when [...] The current rate limiting has been introduced from https://github.com/ExpediaGroup/waggle-dance/pull/314. But when we tested it, we found that it didn't work in krb environment.

Describe the solution you'd like According to the log, users are all empty, as follows . 08:36:28.243 [pool-6-thread-198] INFO com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler - WD Audit:[User:, method:shutdown, args:null] 08:37:15.882 [pool-6-thread-143] INFO com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler - WD Audit:[User:, method:getThreadId, args:null] 08:37:15.883 [pool-6-thread-143] INFO com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler - WD Audit:[User:, method:get_current_notificationEventId, args:null]

After investigation, we found that the current limiting depends on ugi, and the ugi of WD comes from calling the set_ugi method. The call to set_ugi comes from the client of WD. But this client call can only be called in non secure mode (non KRB).

The following code in class HiveMetaStoreClient.java can illustrate : WechatIMG2289

Describe alternatives you've considered Perhaps we need to set up ugi in the KRB environmen.

Additional context For now, we don't need the current limiting. Perhaps I will implement it when needed in the future.

flaming-archer commented 3 days ago

Let me first create a missing feature here...