TobikoData / sqlmesh

Efficient data transformation and modeling framework that is backwards compatible with dbt.
https://sqlmesh.com
Apache License 2.0
1.84k stars 163 forks source link

UI (Docs) throw error with single word quoted identifiers for Snowflake dialect #3379

Closed mpcarter closed 1 week ago

mpcarter commented 1 week ago

This is a really niche bug it seems in the docs module of the UI. It seems benign as plans still function and apply correctly. Just an annoyance. This was on latest version 0.132.1.

If using snowflake dialect, and you have a column with a single word identifier that is quoted with any lowercase characters, for example name as "Name", it throws an error Cannot find column NAME in query.

In this example, the column named "Other" causes the problem. If I do any of the following, the error goes away:

MODEL (
    name example.model_a
    , kind VIEW
    , grain id
    , dialect snowflake
);

select
-- This comment describes the id column
'a' as id
, 1 as amount -- This comment describes the amount column
, 2 as "Other"
;
Traceback (most recent call last):
  File "C:\sqlmesh\Lib\site-packages\web\server\settings.py", line 99, in get_loaded_context
    yield await loop.run_in_executor(
  File "C:\sqlmesh\Lib\site-packages\fastapi\routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\fastapi\routing.py", line 193, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\starlette\concurrency.py", line 42, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\anyio\_backends\_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\anyio\_backends\_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\web\server\api\endpoints\models.py", line 45, in get_model
    return serialize_model(context, model, render_query=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\web\server\api\endpoints\models.py", line 84, in serialize_model
    description = column_description(context, model.name, name)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\sqlmesh\core\lineage.py", line 99, in column_description
    dependencies = column_dependencies(context, model_name, column)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\sqlmesh\core\lineage.py", line 76, in column_dependencies
    for node in lineage(column, model, trim_selects=False).walk():
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\sqlmesh\core\lineage.py", line 63, in lineage
    return sqlglot_lineage(
           ^^^^^^^^^^^^^^^^
  File "C:\sqlmesh\Lib\site-packages\sqlglot\lineage.py", line 118, in lineage
    raise SqlglotError(f"Cannot find column '{column}' in query.")
sqlglot.errors.SqlglotError: Cannot find column 'OTHER' in query.