GlareDB / glaredb

GlareDB: An analytics DBMS for distributed data
https://glaredb.com
MIT License
717 stars 41 forks source link

Bug: Issue with `read_lance` on S3 #3215

Open talagluck opened 2 months ago

talagluck commented 2 months ago

Description

Currently I'm hitting an issue when I try and use read_lance for a Lance table on S3. When I use read_csv on another object in the same bucket passing in credentials in the same way, it works fine.

Here is the error:

---------------------------------------------------------------------------
ExecutionException                        Traceback (most recent call last)
Cell In[16], line 1
----> 1 con.sql(
      2     f"""
      3         SELECT * FROM read_lance(
      4             's3://glaredb-sandbox/lance/table1',
      5             access_key_id => '{aws_access_key_id}',
      6             secret_access_key => '{aws_secret_access_key}',
      7             region =>'us-east-2')
      8     """
      9 ).to_pandas()

ExecutionException: External error: Invalid credentials for s3, got no creds

Command that triggers the error:

con.sql(
    f"""   
        SELECT * FROM read_lance(
            's3://glaredb-sandbox/lance/table1',
            access_key_id => '{aws_access_key_id}',
            secret_access_key => '{aws_secret_access_key}',
            region =>'us-east-2'
        ) LIMIT 10;
    """
).to_pandas()

read_csv command that works:

con.sql(
    f"""   
        SELECT * FROM read_csv(
            's3://glaredb-sandbox/sales_data.csv',
            access_key_id => '{aws_access_key_id}',
            secret_access_key => '{aws_secret_access_key}',
            region =>'us-east-2'
        ) LIMIT 10;
    """
).to_pandas()