aws / aws-sdk-pandas

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
https://aws-sdk-pandas.readthedocs.io
Apache License 2.0
3.92k stars 700 forks source link

athena.read_sql_query() not recognized as a method #1898

Closed markwilber closed 1 year ago

markwilber commented 1 year ago

I ran a query in Athena's Query editor, and obtained the desired result. (It does not seem that the specific query mattered here.) I pasted that working query into a python variable query_string, and then ran:

df = wr.athena.read_sql_query(
     sql=query_string, database="reports_neo", ctas_approach=False
)

I used ctas_approach=False since I do not have write access to the S3 bucket.

Got the following stack trace, which seems ... surprising:

Traceback (most recent call last):
  File "/home/ec2-user/SageMaker/ChampionChallengeModeling/src/./autobuild_voting_models.py", line 1066, in <module>
    rockTheVote()
  File "/home/ec2-user/anaconda3/envs/py3.10.8/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ec2-user/anaconda3/envs/py3.10.8/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ec2-user/anaconda3/envs/py3.10.8/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ec2-user/anaconda3/envs/py3.10.8/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ec2-user/SageMaker/ChampionChallengeModeling/src/./autobuild_voting_models.py", line 1063, in rockTheVote
    vmb.main()
  File "/home/ec2-user/SageMaker/ChampionChallengeModeling/src/./autobuild_voting_models.py", line 1020, in main
    self.update_metastable_doc_ids()
  File "/home/ec2-user/SageMaker/ChampionChallengeModeling/src/./autobuild_voting_models.py", line 626, in update_metastable_doc_ids
    dfnew = self.fetch_QC_UI(
  File "/home/ec2-user/SageMaker/ChampionChallengeModeling/src/./autobuild_voting_models.py", line 530, in fetch_QC_UI
    df = wr.athena.read_sql_query(
  File "/home/ec2-user/anaconda3/envs/py3.10.8/lib/python3.10/site-packages/awswrangler/__init__.py", line 55, in __getattr__
    return getattr(getattr(DynamicInstantiate.__default_session, self._module_name), name)
AttributeError: 'Athena' object has no attribute 'read_sql_query
conda list | grep awswrangler
awswrangler               0.3.1                    pypi_0    pypi

conda list | grep python
ipython_genutils          0.2.0                      py_1    conda-forge
LeonLuttenberger commented 1 year ago

Hey,

Can you please send us the full script, including the imports section?

Also, can you try to update awswrangler to a more recent version? Our latest one is 2.18.0.

Best regards, Leon

jaidisido commented 1 year ago

As Leon said, I think it's simply due to the fact that you are using a very old version (0.3.1) where the Athena module did not exist. Updating to the latest version should resolve this

markwilber commented 1 year ago

Thank you @LeonLuttenberger and @jaidisido for your prompt replies. You had it right: a very stale version, which, oddly enough, conda-forge selected for a fresh install. Supplying an explicit version resolved the issue:

conda install -c conda-forge awswrangler=2.18.*
deeswain commented 2 months ago

I faced this issue a few time. The following steps help me fix the issue.

  1. Make sure boto (or boto3) is installed. -- pip install boto (or pip3 install boto)
  2. Make sure you are using the latest version of awswrangler (as read_sql_query() is part of this library). In my case I was using the awswrangler verion 0.0.8. Upgrading it to the latest version 3.9.0 fixed the issue. -- pip install --upgrade awswrangler (or pip3 install --upgrade awswrangler)