search

BlazingDB / blazingsql

BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
https://blazingsql.com
Apache License 2.0
1.93k stars 183 forks source link

Non-AWS s3 connection #242

Closed zackosmacko32 closed 3 years ago

zackosmacko32 commented 4 years ago

Is your feature request related to a problem? Please describe. Requesting additional plugin to connect to non-aws s3 file store.

Describe the solution you'd like Parameter within the s3 function to enter non-aws s3 URL

felipeblazing commented 4 years ago

@aucahuasi I am looking over how the file systems manage credentials and I see that we are sending them around like this 

    const auto credentials = Aws::Auth::AWSCredentials(accessKeyId, secretKey, sessionToken);
    const RegionResult regionResult = getRegion(bucketName, credentials);

We could change this so that instead we store a shared_ptr that we can make null in the case that credentials were not sent like when a user is using a public bucket. This combined with the update we made to the new SDK should resolve the issues above.

aucahuasi commented 4 years ago

@zackosmacko32 This PR was merged https://github.com/BlazingDB/blazingsql/pull/524 allows non AWS S2 connections, so we can close this issues as soon as you confirm is working fine!

zackosmacko32 commented 4 years ago

Confirmed!

[cid:image001.png@01D612F8.4D6A2680] Zachary Fragoso Manager – Data Science & AI 248.880.8926

From: Percy Camilo Triveño Aucahuasi notifications@github.com Sent: Tuesday, April 14, 2020 11:22 PM To: BlazingDB/blazingsql blazingsql@noreply.github.com Cc: Zack Fragoso (BTB - PF&Ops An) Zachary.Fragoso@dominos.com; Mention mention@noreply.github.com Subject: Re: [BlazingDB/blazingsql] Non-AWS s3 connection (#242)

@zackosmacko32https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zackosmacko32&d=DwMCaQ&c=tuuk6YS91SuQukMhvUXrnA&r=UkcCxc-biD3qxit_bmmip3IeehNcXiuzOi0iHrBil0I&m=hRgj5DkHdATJ_NmY9ZrfCguWvS3UpRJWCM-XCgXJtFw&s=TqVvTlQGtO4irL8FCy6XJ-ytbeFUJOIvp2cuGOefG4A&e= This PR #524https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_BlazingDB_blazingsql_pull_524&d=DwMCaQ&c=tuuk6YS91SuQukMhvUXrnA&r=UkcCxc-biD3qxit_bmmip3IeehNcXiuzOi0iHrBil0I&m=hRgj5DkHdATJ_NmY9ZrfCguWvS3UpRJWCM-XCgXJtFw&s=o6uU2I5kmFB6tQmWg04WHRiogeNILaj7v-vX6q2FRZ0&e= allows non AWS S2 connections, so we can close this issues as soon as you confirm is working fine!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_BlazingDB_blazingsql_issues_242-23issuecomment-2D613793231&d=DwMCaQ&c=tuuk6YS91SuQukMhvUXrnA&r=UkcCxc-biD3qxit_bmmip3IeehNcXiuzOi0iHrBil0I&m=hRgj5DkHdATJ_NmY9ZrfCguWvS3UpRJWCM-XCgXJtFw&s=0Uu8M09ojhrHO9Ki0Ytb6OD7eRGrFvXol9Y9Aia2Q_M&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AJHXFZCUKMI7G2SGF5YIXY3RMUR6XANCNFSM4KHT3WLA&d=DwMCaQ&c=tuuk6YS91SuQukMhvUXrnA&r=UkcCxc-biD3qxit_bmmip3IeehNcXiuzOi0iHrBil0I&m=hRgj5DkHdATJ_NmY9ZrfCguWvS3UpRJWCM-XCgXJtFw&s=MLh8oZuugDzHvFmOHUVOXFwtkDGP_8FngTHUNNASY1Q&e=.