search

aws / aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
http://aws.amazon.com/sdkfornet/
Apache License 2.0
2.06k stars 855 forks source link

PrestoDB query with .NET #1608

Closed sindizzy closed 4 years ago

sindizzy commented 4 years ago

A third-party has provided my company with an AWS Presto database. I can use DBeaver to connect and make simple queries like

select *
from hive.mycompany.trip_master
where yyyymmdd = '20191201' and operator='XY' and carid='3AQQW39''

That all works great but now I need to automate this with a .NET application. Since Athena is based on Presto my first inclination is to use the AWSSDK.Athena library. However there are no examples on how to use the Athena driver to connect to an AWS Presto DB.

Am I going down the right path? Are there examples of .NET code on how to do this? Should I be using the AWSSDK.ElasticMapReduce library?

`

ashishdhingra commented 4 years ago

Hi @sindizzy,

The Athena Code Samples could be helpful. Even though the examples are in Java, it should be easily portable to .NET. Kindly let me know if this is helpful.

Thanks, Ashish

github-actions[bot] commented 4 years ago

This issue has not recieved a response in 2 weeks. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

sindizzy commented 4 years ago

@ashishdhingra Presto is based on Athena but its a different mechanism. I can connect to Athena no problem. Its the Presto db that I am having trouble with. If there is an example specific to Presto I haven't seen it.

ashishdhingra commented 4 years ago

@sindizzy As of now, I could not find any working example on using Presto using .NET SDK. The Athena Code Samples link provides examples in Java. As mentioned earlier, these examples should easily be ported to .NET using the AWSSDK.Athena. Please try porting some examples and let me know if it works.

Thanks, Ashish

sindizzy commented 4 years ago

Thank you, but again that is Athena. Regardless if the sample is in Java, .NET, or Python it is still for Athena. I don't have to port the samples to .NET because Amazon has samples in .NET already. For Athena. I already have working code for connections to an Athena db. Again Athena is not the issue.

The samples I am looking for is Presto which is a different animal.

ashishdhingra commented 4 years ago

Hi @sindizzy,

Good morning.

I was trying to explore for code samples for using Athena with PrestoDB. As per Athena User Guide, the first step is setting up datasource. It doesn't specifically mentions about PrestoDB, may be Athena support could provide some guidance.

Found one useful articles:

As per other information available online, AWS Athena uses Presto in the background. .NET SDK appears to have API for query execution with Athena, with Athena using PrestoDB in the background.

Hope this helps.

Thanks, Ashish

sindizzy commented 4 years ago

I have already gone through all these sources and even posted on the Amazon official forum. There are no samples connecting to presto that I can find.

ashishdhingra commented 4 years ago

@sindizzy Based on the research online, Presto (or PrestoDB) is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. Presto allows querying data where it lives, including Hive, Cassandra, relational databases or even proprietary data stores. A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization.

Amazon EMR and Amazon Athena are the best places to deploy Presto in the cloud, because it does the integration, and testing rigor of Presto for you, with the scale, simplicity, and cost effectiveness of AWS.

The .NET example that you shared demonstrates a simple query against cloudtrail_logs table in Athena.

References:

Not sure on how the 3rd party provided PrestoDB database (not exactly database, it's the query system connected to various data sources) - is it in Athena or EMR? Given that Presto is just a query engine, it should not be confused with the actual source behind the scenes. May be this information from 3rd party could help and Athena support could help with connecting Athena to the source.

Hope this helps.

github-actions[bot] commented 4 years ago

This issue has not recieved a response in 2 weeks. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.