search

JohnOmernik / sqlalchemy-drill

Apache Drill Dialect for SQL Alchemy
Other
53 stars 35 forks source link

Error in DrillDialect #82

Open SatvikNema opened 1 year ago

SatvikNema commented 1 year ago

Hi John, Have installed sqlalchemy-drill version 1.1.2 I am able to connect to my local drill instance.

image

I have created a simple http api endpoint

{
  "type": "http",
  "cacheResults": true,
  "connections": {
    "covid": {
      "url": "https://api.covidtracking.com/v1",
      "requireTail": true,
      "method": "GET",
      "authType": "none",
      "inputType": "json",
      "xmlDataLevel": 1,
      "verifySSLCert": true
    }
  },
  "proxyType": "direct",
  "enabled": true
}

and I can get the data from apache drill's query console by using:

select * from http.covid.`/us/daily.json`

But when I try to create the dataset from apache superset, no tables are loaded.

image

I see the following error on console

Error in DrillDialect_sadrill.get_table_names :: generator raised StopIteration
Error in DrillDialect_sadrill.get_view_names :: generator raised StopIteration

superset version: 2.0.1 python version: 3.8.13 pip version: 23.0.1

jnturton commented 1 year ago

The HTTP storage plugin in Drill doesn't set up any tables.

apache drill> show tables in http;
No rows selected (0.584 seconds)

One option is to create a SQL view in filesystem storage and have Superset discover and query that.

apache drill> create view dfs.tmp.covid_us_daily as select * from http.covid.`/us/daily.json`;
ok       true
summary  View 'covid_us_daily' created successfully in 'dfs.tmp' schema

apache drill (dfs.tmp)> show tables in dfs.tmp;
TABLE_SCHEMA  dfs.tmp
TABLE_NAME    covid_us_daily