SNOW-170140: `location` is not constructed correctly in `write_pandas`

[bmwilly]

Please answer these questions before submitting your issue. Thanks!

1. What version of Python are you using (python --version)?

Python 3.8.2

2. What operating system and processor architecture are you using (python -c 'import platform; print(platform.platform())')?

macOS-10.15.5-x86_64-i386-64bit

3. What are the component versions in the environment (pip list)?

Package                      Version             Location
---------------------------- ------------------- -------------------------------------------------------
agate                        1.6.1
alembic                      1.4.2
amazon-kclpy                 2.0.1
ansiwrap                     0.8.4
appdirs                      1.4.3
appnope                      0.1.0
arrow                        0.15.6
arviz                        0.8.3
asn1crypto                   1.3.0
async-generator              1.10
atpublic                     1.0
attrs                        19.3.0
autograd                     1.3
autograd-gamma               0.4.2
autopep8                     1.5.1
aws-mfa                      0.0.12
awscli                       1.18.75
azure-common                 1.1.25
azure-core                   1.6.0
azure-nspkg                  3.0.2
azure-storage-blob           12.3.2
azure-storage-common         0.37.1
azure-storage-nspkg          3.1.0
b2                           1.4.2
b2sdk                        0.1.8
Babel                        2.8.0
backcall                     0.2.0
binaryornot                  0.4.4
black                        19.10b0
bleach                       3.1.5
boto                         2.49.0
boto3                        1.11.17
botocore                     1.14.17
Brotli                       1.0.7
brotlipy                     0.7.0
cached-property              1.5.1
cachetools                   4.1.0
catboost                     0.22
certifi                      2020.4.5.2
cffi                         1.13.2
cftime                       1.1.3
chardet                      3.0.4
click                        7.1.1
cloudpickle                  1.4.1
cmdstanpy                    0.4.0
colorama                     0.4.3
conda                        4.8.3
conda-package-handling       1.6.0
configobj                    5.0.6
configparser                 5.0.0
convertdate                  2.2.1
convoys                      0.1.7
cookiecutter                 1.7.0
cryptography                 2.9.2
cycler                       0.10.0
Cython                       0.29.20
cytoolz                      0.10.1
dash                         1.12.0
dash-bootstrap-components    0.10.1
dash-core-components         1.10.0
dash-daq                     0.5.0
dash-html-components         1.0.3
dash-renderer                1.4.1
dash-table                   4.7.0
dask                         2.18.1
databricks-cli               0.9.1
decorator                    4.4.2
defusedxml                   0.6.0
distro                       1.4.0
docker                       4.2.1
docker-pycreds               0.4.0
docutils                     0.15.2
dpath                        2.0.1
emcee                        3.0.2
entrypoints                  0.3
ephem                        3.7.7.1
escapism                     1.0.1
fbprophet                    0.6
flake8                       3.7.9
flaky                        3.6.1
Flask                        1.1.2
Flask-Compress               1.5.0
flatten-json                 0.1.7
flufl.lock                   3.2
fsspec                       0.7.4
funcsigs                     1.0.2
funcy                        1.14
future                       0.18.2
futures                      3.1.1
gitdb                        4.0.5
GitPython                    3.1.3
google-api-core              1.17.0
google-api-python-client     1.8.3
google-auth                  1.14.3
google-auth-httplib2         0.0.3
google-auth-oauthlib         0.4.1
googleads                    24.0.0
googleapis-common-protos     1.51.0
gorilla                      0.3.0
grandalf                     0.6
graphviz                     0.13.2
gspread                      3.6.0
gspread-pandas               2.2.3
gunicorn                     20.0.4
holidays                     0.10.2
hologram                     0.0.7
httplib2                     0.17.3
humanize                     2.2.0
idna                         2.9
ijson                        2.6.1
imagecodecs                  2020.5.30
imageio                      2.8.0
importlib-metadata           1.6.1
inflect                      3.0.2
ipdb                         0.12.3
ipykernel                    5.3.0
ipython                      7.15.0
ipython-genutils             0.2.0
ipywidgets                   7.5.1
isodate                      0.6.0
itsdangerous                 1.1.0
jedi                         0.17.0
Jinja2                       2.11.2
jinja2-time                  0.2.0
jmespath                     0.10.0
joblib                       0.15.1
json-rpc                     1.13.0
json5                        0.9.4
jsonpath-ng                  1.5.1
jsonschema                   3.2.0
jupyter-client               6.1.3
jupyter-console              6.1.0
jupyter-core                 4.6.3
jupyter-repo2docker          0.11.0
jupyterlab                   2.1.0
jupyterlab-code-formatter    1.3.1
jupyterlab-server            1.1.5
kafka-python                 1.4.7
keyring                      21.1.1
kiwisolver                   1.2.0
korean-lunar-calendar        0.2.1
leather                      0.3.3
lifelines                    0.24.4
lime                         0.2.0.0
Logbook                      1.5.3
logfury                      0.1.2
LunarCalendar                0.0.9
lxml                         4.5.1
Mako                         1.1.1
MarkupSafe                   1.1.1
matplotlib                   3.2.1
mccabe                       0.6.1
minimal-snowplow-tracker     0.0.2
mistune                      0.8.4
mlflow                       1.7.2
more-itertools               8.4.0
msrest                       0.6.16
mypy                         0.770
mypy-extensions              0.4.3
nanotime                     0.5.2
nb-conda-kernels             2.2.3
nbclient                     0.4.0
nbconvert                    5.6.1
nbdime                       2.0.0
nbformat                     5.0.6
nest-asyncio                 1.3.3
netCDF4                      1.5.3
networkx                     2.4
notebook                     6.0.3
numpy                        1.18.1
oauthlib                     3.1.0
olefile                      0.46
oscrypto                     1.2.0
packaging                    20.4
pandas                       1.0.5
pandocfilters                1.4.2
papermill                    2.1.0
parsedatetime                2.6
parso                        0.7.0
pathspec                     0.8.0
patsy                        0.5.1
pexpect                      4.8.0
pickleshare                  0.7.5
Pillow                       7.1.2
pip                          20.1.1
pipdeptree                   0.13.2
pkginfo                      1.5.0.1
plotly                       4.6.0
pluggy                       0.13.1
ply                          3.11
poyo                         0.5.0
progressbar2                 3.47.0
prometheus-client            0.8.0
prometheus-flask-exporter    0.13.0
prompt-toolkit               3.0.5
protobuf                     3.12.3
psutil                       5.7.0
psycopg2                     2.8.5
psycopg2-binary              2.8.5
ptyprocess                   0.6.0
py                           1.8.2
pyarrow                      0.17.1
pyasn1                       0.4.8
pyasn1-modules               0.2.8
pycodestyle                  2.5.0
pycosat                      0.6.3
pycparser                    2.20
pycryptodomex                3.9.7
pydist-cli                   0.1.1
pydot                        1.4.1
pyflakes                     2.1.1
Pygments                     2.6.1
pygtrie                      2.3.2
pyjavaproperties             0.7
PyJWT                        1.7.1
PyMeeus                      0.3.7
pyOpenSSL                    19.1.0
pyparsing                    2.4.7
pyrsistent                   0.16.0
PySocks                      1.7.1
pystan                       2.19.1.1
pytest                       5.4.1
python-crontab               2.5.1
python-dateutil              2.8.1
python-dotenv                0.13.0
python-editor                1.0.4
python-json-logger           0.1.11
python-slugify               4.0.0
python-utils                 2.3.0
pytimeparse                  1.1.8
pytz                         2020.1
PyWavelets                   1.1.1
PyYAML                       5.3.1
pyzmq                        19.0.1
querystring-parser           1.2.4
readme-renderer              24.0
regex                        2020.6.8
requests                     2.23.0
requests-oauthlib            1.3.0
requests-toolbelt            0.9.1
retrying                     1.3.3
rsa                          3.4.2
ruamel-yaml                  0.15.80
ruamel.yaml                  0.16.10
ruamel.yaml.clib             0.2.0
s3fs                         0.4.2
s3transfer                   0.3.3
scikit-image                 0.17.2
scikit-learn                 0.22.2.post1
scipy                        1.4.1
seaborn                      0.10.0
semver                       2.9.1
Send2Trash                   1.5.0
setuptools                   47.3.1.post20200616
setuptools-git               1.2
shap                         0.35.0
shortuuid                    1.0.1
simplejson                   3.17.0
six                          1.15.0
smmap                        3.0.4
snowflake-connector-python   2.2.7
snowflake-sqlalchemy         1.2.1
SQLAlchemy                   1.3.17
sqlparse                     0.3.1
statsmodels                  0.11.1
tabulate                     0.8.7
tenacity                     6.2.0
terminado                    0.8.3
testpath                     0.4.4
text-unidecode               1.3
texttable                    1.6.2
textwrap3                    0.9.2
tifffile                     2020.6.3
toml                         0.10.1
toolz                        0.10.0
torch                        1.4.0
torchvision                  0.5.0
tornado                      6.0.4
tqdm                         4.46.1
traitlets                    4.3.3
treelib                      1.6.1
twine                        3.1.1
typed-ast                    1.4.1
typing-extensions            3.7.4.2
uritemplate                  3.0.1
urllib3                      1.25.9
voluptuous                   0.11.7
wcwidth                      0.2.4
webencodings                 0.5.1
websocket-client             0.57.0
Werkzeug                     1.0.1
wheel                        0.34.2
whichcraft                   0.6.1
widgetsnbextension           3.5.1
xarray                       0.15.1
xmltodict                    0.12.0
zc.lockfile                  2.0
zeep                         3.4.0
zipp                         3.1.0

4. What did you do?

There are many cases where the location (https://github.com/snowflakedb/snowflake-connector-python/blob/master/pandas_tools.py#L87) is not constructed correctly in write_pandas. Take three examples that are all possible and incorrect:

The result is that the user always need to do something like

conn.execute(f"use database {db}")
conn.execute(f"create schema if not exists {schema}")
conn.execute(f"use schema {schema}")
conn.execute(f"create table if not exists {table}")

to use pd_writer / write_pandas.

[sfc-gh-mkeller]

I made a mistake the location should be constructed in this manner:

location = ((('"' + database + '".') if database else '') +
    (('"' + schema + '".') if schema else '') +
    ('"' + table_name + '"'))

My apologies, I'll fix this for next release! Until then you could patch this into local install!

[bmwilly]

@sfc-gh-mkeller great, thanks!

[adde-rishimehta]

@sfc-gh-mkeller When is this gonna release?

[sfc-gh-mkeller]

It should have been included in 2.2.9 @adde-rishimehta

[adde-rishimehta]

Thanks @sfc-gh-mkeller , should be good to close this Issue then.