sqlalchemy.exc.DBAPIError: Error when creating a DuckDB Connection

cnstlungu commented 1 year ago

I'm trying to connect Superset to a DuckDB database file. I receive the following error:

SupersetErrorsException
Traceback (most recent call last):
  File "/app/superset/databases/commands/test_connection.py", line 120, in run
    raise DBAPIError(None, None, None)
sqlalchemy.exc.DBAPIError: (builtins.NoneType) None
(Background on this error at: http://sqlalche.me/e/13/dbapi)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 113, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 110, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1507, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 245, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 83, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/databases/api.py", line 709, in test_connection
    TestConnectionDatabaseCommand(g.user, item).run()
  File "/app/superset/databases/commands/test_connection.py", line 145, in run
    raise DatabaseTestConnectionFailedError(errors) from ex
superset.databases.commands.exceptions.DatabaseTestConnectionFailedError: [SupersetError(message='(builtins.NoneType) None\n(Background on this error at: http://sqlalche.me/e/13/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'DuckDB', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
2023-03-31 10:57:26,232:WARNING:superset.views.base:SupersetErrorsException
Traceback (most recent call last):
  File "/app/superset/databases/commands/test_connection.py", line 120, in run
    raise DBAPIError(None, None, None)
sqlalchemy.exc.DBAPIError: (builtins.NoneType) None
(Background on this error at: http://sqlalche.me/e/13/dbapi)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 113, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 110, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1507, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 245, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 83, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/databases/api.py", line 709, in test_connection
    TestConnectionDatabaseCommand(g.user, item).run()
  File "/app/superset/databases/commands/test_connection.py", line 145, in run
    raise DatabaseTestConnectionFailedError(errors) from ex
superset.databases.commands.exceptions.DatabaseTestConnectionFailedError: [SupersetError(message='(builtins.NoneType) None\n(Background on this error at: http://sqlalche.me/e/13/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'DuckDB', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

How to reproduce the bug

Go to Data => Databases
Click + Database and Select DuckDB from the dropdown list
Enter 'duckdb:///app/tutorial.db'
See error popup : An error occurred while creating databases: Connection failed, please check your connection settings

Also, when creating the engine with the superset CLI

superset set_database_uri -d duckdb -u duckdb:///app/tutorial.duckdb

The same None Error Type appears.

Expected results

A DuckDB connection would be successfully create.

Actual results

An error is popped-up : An error occurred while creating databases: Connection failed, please check your connection settings

Screenshots

Environment

(please complete the following information):

browser type and version: Firefox 111.0 (64-bit)
superset version: Superset 2.0.0 built from superset docker image
python version: Python 3.8.12
node.js version: node -v
any feature flags active:

FROM apache/superset:2.0.0 

ARG SUPERSET_ADMIN
ARG SUPERSET_PASSWORD
ARG POSTGRES_USER
ARG POSTGRES_PASSWORD
# Switching to root to install the required packages
USER root
COPY --chown=superset:superset ./superset/assets/import .
RUN pip install psycopg2-binary
RUN pip install duckdb-engine
RUN pip install duckdb
USER superset
RUN superset fab create-admin \
              --username ${SUPERSET_ADMIN} \
              --firstname Superset \
              --lastname Admin \
              --email admin@example.com \
              --password ${SUPERSET_PASSWORD}
RUN superset db upgrade
RUN superset init
RUN superset set_database_uri -d OLTP  -u postgresql+psycopg2://${POSTGRES_USER}:${POSTGRES_PASSWORD}@oltp:5432/sales_oltp
RUN superset set_database_uri -d DW  -u postgresql+psycopg2://${POSTGRES_USER}:${POSTGRES_PASSWORD}@dw:5432/sales_dw
RUN superset import-dashboards -p ./dashboard.json

aiohttp==3.8.1
aiosignal==1.2.0
alembic==1.6.5
amqp==5.1.0
# Editable install with no version control (apache-superset==2.0.0)
-e /app
apispec==3.3.2
appnope==0.1.3
asttokens==2.0.5
async-timeout==4.0.2
attrs==21.2.0
Babel==2.9.1
backcall==0.2.0
backoff==1.11.1
billiard==3.6.4.0
bleach==3.3.1
boto3==1.18.19
botocore==1.21.19
Brotli==1.0.9
cached-property==1.5.2
cachelib==0.4.1
celery==5.2.2
certifi==2021.10.8
cffi==1.14.6
chardet==4.0.0
charset-normalizer==2.0.4
click==8.0.4
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.4
convertdate==2.3.2
cron-descriptor==1.2.24
croniter==1.0.15
cryptography==3.4.7
decorator==5.1.1
deprecation==2.1.0
dnspython==2.1.0
duckdb==0.7.1
duckdb-engine==0.7.0
email-validator==1.1.3
et-xmlfile==1.1.0
executing==0.8.3
Flask==2.0.3
Flask-AppBuilder==4.1.3
Flask-Babel==1.0.0
Flask-Caching==1.10.1
Flask-Compress==1.10.1
Flask-Cors==3.0.10
Flask-JWT-Extended==4.3.1
Flask-Login==0.4.1
Flask-Migrate==3.1.0
Flask-SQLAlchemy==2.5.1
flask-talisman==0.8.1
Flask-WTF==0.14.3
frozenlist==1.3.0
func-timeout==4.3.5
future==0.18.2
geographiclib==1.52
geopy==2.2.0
graphlib-backport==1.0.3
gunicorn==20.1.0
hashids==1.3.1
holidays==0.10.3
humanize==3.11.0
idna==3.2
ijson==3.1.4
ipython==8.3.0
isodate==0.6.0
itsdangerous==2.1.1
jedi==0.18.1
Jinja2==3.0.3
jmespath==0.10.0
jsonlines==2.0.0
jsonschema==3.2.0
kombu==5.2.4
korean-lunar-calendar==0.2.1
linear-tsv==1.1.0
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==2.0.1
marshmallow==3.13.0
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.23.1
matplotlib-inline==0.1.3
msgpack==1.0.2
multidict==5.1.0
mysqlclient==2.1.0
numpy==1.22.1
openpyxl==3.0.7
packaging==21.3
pandas==1.3.4
parsedatetime==2.6
parso==0.8.3
pexpect==4.8.0
pgsanity==0.2.9
pickleshare==0.7.5
Pillow==9.1.0
polyline==1.4.0
prison==0.2.1
progress==1.6
prompt-toolkit==3.0.28
psycopg2-binary==2.9.1
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
pyarrow==5.0.0
pycparser==2.20
pydruid==0.6.2
Pygments==2.12.0
PyHive==0.6.5
pyinstrument==4.0.2
PyJWT==2.4.0
PyMeeus==0.5.11
pyparsing==3.0.6
pyrsistent==0.16.1
python-dateutil==2.8.2
python-dotenv==0.19.0
python-editor==1.0.4
python-geohash==0.8.5
pytz==2021.3
PyYAML==5.4.1
redis==3.5.3
requests==2.26.0
rfc3986==1.5.0
s3transfer==0.5.0
sasl==0.3.1
selenium==3.141.0
simplejson==3.17.3
six==1.16.0
slackclient==2.5.0
SQLAlchemy==1.3.24
SQLAlchemy-Utils==0.37.8
sqloxide==0.1.17
sqlparse==0.3.0
stack-data==0.2.0
tableschema==1.20.2
tabulate==0.8.9
tabulator==1.53.5
thrift==0.13.0
thrift-sasl==0.4.3
traitlets==5.2.1.post0
typing-extensions==3.10.0.0
unicodecsv==0.14.1
urllib3==1.26.6
vine==5.0.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.3
WTForms==2.3.3
WTForms-JSON==0.3.3
xlrd==2.0.1
yarl==1.6.3

Checklist

Make sure to follow these steps before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Add any other context about the problem here.

cnstlungu commented 1 year ago

Please disregard this issue.

The problem was that it was the provided path, which I provided incorrectly.

raj95 commented 1 year ago

I'm running into the same issue.. i'm giving the absolute path of the .db file.. I'm getting this error, Can anyone help here:

SQL ALCHEMY URI: duckdb:////Users/rajashekarpantangi/Desktop/duckdb/databases/test.db

ERROR: (builtins.NoneType) None [SQL: (duckdb.IOException) IO Error: Cannot open file "/Users/rajashekarpantangi/Desktop/duckdb/databases/test.db": No such file or directory

caohaisil201 commented 1 year ago

Any update here, I have the same issue with @raj95

a-bartoszek commented 1 year ago

I am also having a similar error. I'm trying to connect from Superset in a docker container to a local duckdb database. I tried to reference the instructions at the bottom of this page, but no luck... https://superset.apache.org/docs/installation/installing-superset-using-docker-compose/

apache / superset