AttributeError: module 'dbt.flags' has no attribute 'PROFILES_DIR' with dbt 1.5

[johnpkennedy]

Search before asking

• [X] I searched the issues and found no similar issues.

What Happened

After upgrading to dbt 1.5 and SQLFluff 2.1.0, I get the following error when I run sqlfluff lint:

AttributeError: module 'dbt.flags' has no attribute 'PROFILES_DIR'

Expected Behaviour

I don't get this error on SQLFluff 2.0.7 with dbt 1.4.6.

Observed Behaviour

% sqlfluff lint
=== [dbt templater] Sorting Nodes...                                                                     
file /Users/johnkennedy/Projects/maple-data/dbt/dbt_packages/audit_helper/integration_tests/analyses/compTraceback (most recent call last):
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 605, in lint
    result = lnt.lint_paths(
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1206, in lint_paths
    for i, linted_file in enumerate(runner.run(expanded_paths, fix), start=1):
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 106, in run
    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 59, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 42, in iter_rendered
    for fname in self.linter.templater.sequence_files(
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff_templater_dbt/templater.py", line 263, in sequence_files
    self.profiles_dir = self._get_profiles_dir()
  File "/Users/johnkennedy/.local/share/virtualenvs/dbt--3QRDvUu/lib/python3.10/site-packages/sqlfluff_templater_dbt/templater.py", line 196, in _get_profiles_dir
    or flags.PROFILES_DIR
AttributeError: module 'dbt.flags' has no attribute 'PROFILES_DIR'

How to reproduce

1. Upgrade dbt to 1.5
2. Upgrade SQLFluff to 2.1.0
3. Run sqlfluff lint

Dialect

redshift

Version

sqlfluff - 2.1.0 sqlfluff-templater-dbt - 2.1.0 Python 3.10.11

Configuration

[sqlfluff] dialect = redshift templater = dbt

[sqlfluff:templater:dbt] apply_dbt_builtins = true profile = maple project_dir = dbt

Are you willing to work on and submit a PR to address the issue?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[the-chris-mitchell]

Same issue here.

dbt: 1.5.0
sqlfluff: 2.1.0
sqlfluff-templater-dbt: 2.1.0
python: 3.10.8

[the-chris-mitchell]

As a temporary workaround I added this to .sqlfluff:

[sqlfluff:templater:dbt]
profiles_dir = .

[WittierDinosaur]

Ah, you're right, we missed a reference. We'll try to get a patch out shortly

[piankris]

I have the same issue. I prefer to not specify profiles_dir to default to the profiles.yml in my home directory.

From the docs:

If the profiles_dir setting is omitted, SQLFluff will look for the profile in the default location, which varies by operating system. On Unix-like operating systems (e.g. Linux or macOS), the default profile directory is ~/.dbt/.

While it worked before, it doesn't work now. dbt version: 1.3.1 sqlfluff version: 2.0.7 and tried 2.1.0

[nathangriffiths-cdx]

Related issue in dbt core, quite possibly the fix for this in dbt will resolve this issue as well: https://github.com/dbt-labs/dbt-core/issues/7511

[judahrand]

@WittierDinosaur Still planning to release a patch version?

[netragourlay]

I'm having the same issue, and these are my versions Python 3.11.0 dbt-core==1.5.0 dbt-bigquery==1.5.1 sqlfluff-templater-dbt==2.0.7

[mariana-s-fernandes]

Also having this issue with Python 3.9.16 dbt-core==1.5.0 dbt-bigquery==1.5.1 sqlfluff-templater-dbt==2.0.7

[elyobo]

https://github.com/sqlfluff/sqlfluff/pull/4828 was possibly related but doesn't appear to have resolved the issue affecting many here and in the comments there.

[nathangriffiths-cdx]

I am also getting this error: Python: 3.10.10 dbt-core: 1.5.0 dbt-bigquery: 1.5.1 sqlfluff: 2.1.0 sqlfluff-templater-dbt: 2.1.0

sqlfluff lint ./path/to/my/model.sql

=== [dbt templater] Sorting Nodes...
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\path\to\dbt\project\venv\Scripts\sqlfluff.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff\cli\commands.py", line 605, in lint
    result = lnt.lint_paths(
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff\core\linter\linter.py", line 1206, in lint_paths
    for i, linted_file in enumerate(runner.run(expanded_paths, fix), start=1):
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff\core\linter\runner.py", line 106, in run
    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff\core\linter\runner.py", line 59, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff\core\linter\runner.py", line 42, in iter_rendered
    for fname in self.linter.templater.sequence_files(
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff_templater_dbt\templater.py", line 263, in sequence_files
    self.profiles_dir = self._get_profiles_dir()
  File "C:\Users\path\to\dbt\project\venv\lib\site-packages\sqlfluff_templater_dbt\templater.py", line 196, in _get_profiles_dir
    or flags.PROFILES_DIR
AttributeError: module 'dbt.flags' has no attribute 'PROFILES_DIR'

[nathangriffiths-cdx]

My python isn't up to fixing this but the issue seems to be this code in templater.py:

        dbt_profiles_dir = os.path.abspath(
            os.path.expanduser(
                self.sqlfluff_config.get_section(
                    (self.templater_selector, self.name, "profiles_dir")
                )
                or flags.PROFILES_DIR
            )
        )

This expects a "profiles_dir" value to be provided from either sqlfluff config OR dbt's internal flags - however, looking at the code for dbt's flags it looks like it's possible for the profiles_dir flag to not be set (or set to None).

If "profiles_dir" isn't set in sqlfluff config then it tries to use PROFILES_DIR from dbt flags, however if this flag isn't set inside dbt (or is set to None?) then flags will have no PROFILES_DIR attribute and the above error is generated.

[greg-finley]

fixed via https://github.com/sqlfluff/sqlfluff/pull/4886

[elyobo]

The same error message is occurring again with dbt 1.8.0 released today (using sqlfluff 3.0.6)

[elyobo]

The same error message is still occurring with dbt 1.8.1 using the sqlfluff 3.0.7 release, so it seems like it's unfortunately not resolving all the 1.8 issues.

[Startouf]

I had troubles similar to this issue after a DBT upgrade. It seems like https://github.com/dbt-labs/dbt-core/issues/10135#issuecomment-2113728550 helped me resolve the issue, at least on my local machine (and the broken CI/github actions seem to be fixed too)

Quoting :

python -m pip uninstall -y dbt-adapters
python -m pip install --upgrade dbt-core dbt-common dbt-adapters
dbt --version
# The pip uninstall -y dbt-adapters is the important part.

Our project uses poetry so I also re-ran a poetry install to make sure everything was right

[elyobo]

Thanks, those instructions didn't work for me (using poetry), but the comments hinted at the fix - blowing away the virtual env and starting again sorted it for me.

[hc1wwilger]

Adding the addition to the config for the . directory didn't work for me, however, blowing away the venv did.

sqlfluff, version 3.1.1