search

airbnb / airpal

Web UI for PrestoDB.
http://airbnb.github.io/airpal
Apache License 2.0
2.76k stars 458 forks source link

Airpal preview features run incorrect queries against Presto #171

Open danielcorin opened 8 years ago

danielcorin commented 8 years ago

I installed Airpal using ./gradlew clean shadowJar and have been running it using

java -server \
     -Duser.timezone=UTC \
     -cp build/libs/airpal-*-all.jar com.airbnb.airpal.AirpalApplication server reference.yml

My reference.yml is:

# Logging settings
logging:

  loggers:
    org.apache.shiro: WARN

  # The default level of all loggers. Can be OFF, ERROR, WARN, INFO, DEBUG, TRACE, or ALL.
  level: WARN

# HTTP-specific options.
server:
  applicationConnectors:
    - type: http
      port: 8081
      idleTimeout: 10 seconds

  adminConnectors:
    - type: http
      port: 8082

shiro:
  iniConfigs: ["classpath:shiro_allow_all.ini"]

dataSourceFactory:
  driverClass: com.mysql.jdbc.Driver
  user: root
  password: pass
  url: jdbc:mysql://127.0.0.1:3306/airpal

flywayFactory:
  locations: ["classpath:db.migration.common", "classpath:db.migration.mysql"]

# The URL to the Presto coordinator.
prestoCoordinator: <coordinator_url>:8080/

prestoUser: dan
prestoCatalog: hive
prestoSchema: default

When I load Airpal at <coordinator_url>:8081, Airpal runs the following query against Presto

SELECT table_catalog, table_schema, table_name FROM information_schema.tables WHERE table_catalog = 'hive'

I can click on the text field to select a table and the available tables in the hive.default space are displayed. However, when I click a table, Airpal run the incorrect queries against Presto.

For example, when I click default.test_view, Airpal runs the following three queries in this order (top down):

SHOW PARTITIONS FROM default.test_view
SHOW COLUMNS FROM default.test_view
SELECT * FROM hive.default.test_view LIMIT 100

Since the first two queries don't use hive as the catalog, they fail and Airpal can't get the data it needs to show the columns of the table or create a data preview. Below is the stack trace from Airpal but I suspect the problem stems from the incorrect queries.

ERROR [2016-03-24 22:56:02,697] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 3371e9ff4ea1cde3
! java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean
! at com.airbnb.airpal.presto.metadata.ColumnCache$4.apply(ColumnCache.java:141) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.ColumnCache$4.apply(ColumnCache.java:132) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.core.execution.QueryClient.executeWith(QueryClient.java:45) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.ColumnCache.queryColumns(ColumnCache.java:132) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.ColumnCache.access$000(ColumnCache.java:35) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.ColumnCache$1.load(ColumnCache.java:57) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.airbnb.airpal.presto.metadata.ColumnCache$1.load(ColumnCache.java:53) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) ~[airpal-0.1.0-SNAPSHOT-all.jar:na]
! ... 74 common frames omitted
yanyani989 commented 8 years ago

Looking at the class responsible for listing columns at: https://github.com/airbnb/airpal/blob/master/src/main/java/com/airbnb/airpal/presto/metadata/ColumnCache.java#L141 It looks like it is expecting the "SHOW COLUMNS FROM $table" query to return isNullable and isPartition on 3rd and 4th columns, but running manually via presto jdbc this query is only returning column name, type and comment.

I tried editing those lines to get around that error but then encounter other dataType related errors (e.g. decimal support was just added to presto, and the type signature decimal(38,0) would cause an exception: java.lang.IllegalArgumentException: Bad characters in base type: decimal(38,0) with the com.facebook.presto.spi.type.TypeSignature.parseTypeSignature(String) logic).

And it doesn't appear to be a straightforward upgrade to latest prestoVersion (via -Dairpal.prestoVersion or build.gradle) as airpal's interface with presto-spi no longer works on newer presto versions, e.g. com.facebook.presto.client.Column now requires ClientTypeSignature in its constructor so https://github.com/airbnb/airpal/blob/master/src/main/java/com/airbnb/airpal/presto/metadata/ColumnCache.java#L140 will break without change.

dvkalra commented 8 years ago

do we have an update in this issue? im having the same trouble.. and would love to get the preview feature working.

surendraGaddam99 commented 8 years ago

image

Presto provides some predefined queries which can be used for getting list of tables ,columns etc. This information is available under presto release documentation.

Thanks Surendra

dletendre commented 7 years ago

I think this is the same issue as #172

I'm seeing this same error with latest Airpal. I can run a simple query, but when I choose a table, the columns don't show and this error shows up (I'm using Presto 0.166) ERROR [2017-02-24 17:48:56,973] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: c6dd80734466a023 ! java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean

sbourke commented 7 years ago

I'm also experiencing a similar problem when installing a fresh instance from master.

akyekth commented 7 years ago

I Have the same problem, got the same error io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: a07a0ab8fd95dee5 ! java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Boolean ! at com.airbnb.airpal.presto.metadata.ColumnCache$4.apply(ColumnCache.java:143)

It would be great any one help to solve this issue.

Thanks Aruna

fghorbel commented 6 years ago

Use this repository instead. It has the fix. https://github.com/StunlockStudios/airpal/archive/master.zip

The github: https://github.com/StunlockStudios/airpal/