dbt-osmosis breaks sqlfluff linting

[FrankTub]

I have an virtual python environment with python version 3.10.6.

My requirements.txt looks as following.

dbt-core==1.3.2
dbt-postgres==1.3.2
sqlfluff==1.4.5
sqlfluff-templater-dbt==1.4.5
dbt-osmosis==0.9.8

When I now run a sqlfluff lint in that python environment I get below error.

=== [dbt templater] Sorting Nodes...
Traceback (most recent call last):
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 582, in lint
    result = lnt.lint_paths(
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1164, in lint_paths
    for i, linted_file in enumerate(runner.run(expanded_paths, fix), start=1):
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 106, in run
    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 59, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 42, in iter_rendered
    for fname in self.linter.templater.sequence_files(
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff_templater_dbt/templater.py", line 280, in sequence_files
    for key, node in self.dbt_manifest.nodes.items():
  File "/Users/frank.tubbing/.asdf/installs/python/3.10.6/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/sqlfluff_templater_dbt/templater.py", line 154, in dbt_manifest
    self.dbt_manifest = ManifestLoader.get_full_manifest(self.dbt_config)
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/dbt/parser/manifest.py", line 206, in get_full_manifest
    adapter = get_adapter(config)  # type: ignore
  File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.10/site-packages/dbt_osmosis/core/osmosis.py", line 12, in <lambda>
    dbt.adapters.factory.get_adapter = lambda config: config.adapter
AttributeError: 'RuntimeConfig' object has no attribute 'adapter'

While if I deinstall dbt-osmosis in that virtual environment sqlfluff works as it should. Could someone please take a look why this breaks the code?

[awoehrl]

Hi! I seeing the same problem.

[z3z1ma]

Took a first stab at a release, leveraging vendored interface that should be compatible with everything. New version is 0.10.0 on pypi. Exclusively focused on the yaml commands for now. Gutted sqlfluff stuff so it should be fine now.

[FrankTub]

@z3z1ma , thanks for picking this up!! Just tried it out and when I now run below command my sqlfluff version is automatically downgraded.

pip install dbt-osmosis

The output looks something like:

Collecting dbt-osmosis
  Using cached dbt_osmosis-0.10.0-py3-none-any.whl (103 kB)
Requirement already satisfied: dbt-core>=1 in ./venv/lib/python3.10/site-packages (from dbt-osmosis) (1.4.5)
Collecting fastapi<0.86.0,>=0.85.0
...
Installing collected packages: bottle, websockets, uvloop, sniffio, smmap, orjson, httptools, h11, uvicorn, gitdb, anyio, watchfiles, starlette, sqlfluff, GitPython, fastapi, dbt-osmosis
  Attempting uninstall: sqlfluff
    Found existing installation: sqlfluff 2.0.0
    Uninstalling sqlfluff-2.0.0:
      Successfully uninstalled sqlfluff-2.0.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sqlfluff-templater-dbt 2.0.0 requires sqlfluff==2.0.0, but you have sqlfluff 1.4.5 which is incompatible.
Successfully installed GitPython-3.1.31 anyio-3.6.2 bottle-0.12.25 dbt-osmosis-0.10.0 fastapi-0.85.2 gitdb-4.0.10 h11-0.14.0 httptools-0.5.0 orjson-3.8.7 smmap-5.0.0 sniffio-1.3.0 sqlfluff-1.4.5 starlette-0.20.4 uvicorn-0.18.3 uvloop-0.17.0 watchfiles-0.18.1 websockets-10.4

Now it still breaks my virtualenv. Could you please take another look?

[z3z1ma]

@FrankTub ah yes. Need the poetry declared deps to catch up with what I gutted. I will push an update on an hour or so that should address that.