search

awslabs / aws-athena-query-federation

The Amazon Athena Query Federation SDK allows you to customize Amazon Athena with your own data sources and code.
Apache License 2.0
557 stars 293 forks source link

[QUESTION] Glue Catalog Table for Amazon TimeStream #266

Closed ashpad closed 3 years ago

ashpad commented 4 years ago

I have a question that I don't think is related to a bug or feature request.

Per My understanding on discussions with Anthony for timestream glue integration :

This needs to be done in case of Timestream db/tables having mixed case names

  1. We need to create a glue Catalog DB and Table name exactly as the same name as the Timestream DB table ?

  2. It says Glue View--> are you using terminology of GLue table and athena view interchangably?

3.Are you able to share a sample create table script ? with all parameters ? (Screenshot is good but its hard to understad how you arrived at it)

  1. I/O format is set as parquet.i guess it is ignored?
avirtuos commented 4 years ago

seems like we can help resolve this issue, at-least partially by enhancing the timestream connector to have something similar to what our other case sensitive connectors use: https://github.com/awslabs/aws-athena-query-federation/blob/master/athena-cloudwatch/src/main/java/com/amazonaws/athena/connectors/cloudwatch/CloudwatchTableResolver.java

avirtuos commented 4 years ago

2 the readme has exmaples of this but basically Glue Tables support a OriginalViewText field. If you put SQL in there, the connector will treat the Table as view. Additionally if you set a table property of _viewmetadata on your table and put SQL in it, the connector will also treat that table as a view.

3 I dont have one handy but ill try to make one soon. There is a screenshot of a sample table you can create from the Glue console in the ReadMe.

4 anything other than the columns and the _viewmetadata table property are ignored so you can set them to anything you like ... or leave them empty if the console allows it. This is a bit of a rough edge because the Glue Console does validations as if the table is a Hive Table and in the case of our connector...we aren't really using it like Hive.

avirtuos commented 3 years ago

I'm closing this issue due to lack of activity, please reopen if you have further questions