Runtime errors in VSCode

[alejobs]

Hi,

I'm running this extension in VSCode together with Dev containers extension (see: https://code.visualstudio.com/docs/remote/containers).

The configuration I have for this extension is:

{ "terminal.integrated.defaultProfile.linux#": "/bin/bash", "python.pythonPath": "/usr/local/bin/python", "files.associations": { "*.sql": "jinja-sql" }, "sql.format.enable": false, "sqlfluff.config": "${workspaceFolder}/.sqlfluff", "sqlfluff.dialect": "snowflake", "sqlfluff.executablePath": "/usr/local/bin/sqlfluff", "sqlfluff.format.enabled": true, "sqlfluff.format.workingDirectory": "", "sqlfluff.ignoreLocalConfig": false, "sqlfluff.ignoreParsing": true, "sqlfluff.linter.run": "onSave", "sqlfluff.experimental.format.executeInTerminal": false, "editor.formatOnSave": false, "[sql]": { "editor.defaultFormatter": "dorzey.vscode-sqlfluff" } }

Whenever I try to format a SQL file I get: WARNING File /dbt/stdin was not found in dbt project

And then this code gets added to the beginning of the SQL file: 16:32:27 [WARNING]: Deprecated functionality Thesource-pathsconfig has been renamed tomodel-paths. Please update your dbt_project.ymlconfiguration to reflect this change. 16:32:27 [WARNING]: Deprecated functionality Thedata-pathsconfig has been renamed toseed-paths. Please update your dbt_project.ymlconfiguration to reflect this change.

[RobertOstermann]

Are you using the DBT templater? If so try changing sqlfluff.experimental.format.executeInTerminal to True

[alejobs]

Yes, I'm using DBT templater. Here is my .sqlfluff config file contents:

`[sqlfluff] dialect = snowflake templater = dbt exclude_rules = L001, L034

[sqlfluff:templater:dbt] project_dir = ./

[sqlfluff:rules] max_line_length = 999 tab_space_size = 4 indent_unit = space

[sqlfluff:rules:L010]

Keywords

capitalisation_policy = lower

[sqlfluff:rules:L030]

Function names

capitalisation_policy = lower

[sqlfluff:rules:L040]

Null & Boolean Literals

capitalisation_policy = lower

[sqlfluff:rules:L057]

Special Characters in Identifiers

allow_space_in_identifier = True additional_allowed_characters = ['.','(',')','-']`

If I change the parameter you mention, then I get this error:

/usr/local/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 18 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

[RobertOstermann]

Does the formatting/linting work after changing that setting?

If so I would recommend setting the "sqlfluff.suppressNotifications" to true. Sometime the extension shows extra errors that really don't matter. You can still see the errors in the output channel.

Also here are a the settings you need for dbt templater to work.

"sqlfluff.linter.run": "onSave", "sqlfluff.experimental.format.executeInTerminal": true, "editor.formatOnSave": false, // You can set this to just sql if you want.

[alejobs]

I configured all these params but still does not work...

[RobertOstermann]

In the output channels there should be a SQLFluff channel. Can you go to that channel then run the format command. The output channel should show you the exact command that is being run. Copy that command and try running it in the terminal (might have to be git bash terminal, but I'm not sure on that). If the formatting from the extension messed up your file you should undo those changes first. Let me know what happens when you run that format command in the terminal, I want to make sure this is an issue with the extension and not a SQLFluff issue.

[tnightengale]

I also have this exact issue. It would be great if this extension eventually worked for VS Code remote containers, with dbt...

[slve]

@RobertOstermann thanks for dealing w/this issue, here's output from the sqlfluff output channel

------------------------------------------------------------

Reading from stdin, not file, input may be dirty/partial

--------------------Executing Command--------------------

[redacted]/bin/sqlfluff lint --format json --config [redacted]/.sqlfluff --dialect bigquery --exclude-rules L009,L016 -

------------------------------------------------------------

Received close event, code 0 signal null
Raw stdout output:

------------------------------------------------------------

==== readout ====
==== summary ====
violations:        0 status:         PASS
[]

------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

WARNING    File [redacted]/stdin was not found in dbt project 

of course if you echo an empty string into sqlfluff format, it wont' be able to do much, yet it's complaining

   echo '' | sqlfluff lint --format json -
==== readout ====

WARNING    File [redacted]/stdin was not found in dbt project
==== summary ====
violations:        0 status:         PASS
[]

and this is the exact error I also keep getting here's my configuration

    "sqlfluff.config": "${workspaceFolder}/.sqlfluff",
    "sqlfluff.dialect": "bigquery",
    "sqlfluff.excludeRules": ["L009","L016"],
    "sqlfluff.executablePath": "[redacted]/bin/sqlfluff",
    "sqlfluff.experimental.format.executeInTerminal": true,
    "sqlfluff.format.enabled": true,
    "sqlfluff.format.workingDirectory": "",
    "sqlfluff.ignoreLocalConfig": false,
    "sqlfluff.ignoreParsing": false,
    "sqlfluff.linter.run": "onType",
    "sqlfluff.rules": [],

thank you!

[Luttik]

I am having the exact same issue

[RobertOstermann]

@slve Are you also using a dbt templater? I don't think the dbt templater allow for linting using stdin. Can you try changing the setting "sqlfluff.linter.run" to "onSave" and let me know if that changes the output and what the new output is.

[Luttik]

@RobertOstermann For me this resolves the exceptions. However, it would be great if it is actually ran onType. Or at least also when we open a new file.

[RobertOstermann]

@Luttik I am not able to change this to allow it to run onType, you would have to create an issue with the actual SQLFluff repo for that I think. Currently, the dbt-templater does not allow me to pass in the file contents through stdin. It only allows me to pass the path of a file, which means if the file is unsaved it will not lint correctly.

I have done some work on another dbt templater that uses dbt-osmosis to lint the files. That templater does not allow for formatting the document, but does lint much faster than the regular dbt templater and allows for linting onType. Would you be interested in me publishing the changes I have made to allow that templater to work?

[Luttik]

Hi Robert, That might be a good suggestion...

I think that stability is key though for an IDE plugin. So if it is necessary, it might be a good fix to just enforce the onSave behavior when the templater is DBT (and maybe send a VSCode popup warning when first detected).

[slve]

@RobertOstermann yes, I am using dbt templater. Thanks using onSave works fine!

[RobertOstermann]

Hi Robert, That might be a good suggestion...

I think that stability is key though for an IDE plugin. So if it is necessary, it might be a good fix to just enforce the onSave behavior when the templater is DBT (and maybe send a VSCode popup warning when first detected).

Yeah, I could do that when the configuration file path is provided. The difficulty comes when that path is not provided for this extensions settings, then I rely on the SQLFluff tool to find the configuration file and do not know if the templater is dbt or not.

I have also thought about adding a sqlfluff.dbt.enabled setting that would automatically use the settings required for dbt to work, such as onSave, so you wouldn't have to change each of those settings and it would be only 1 setting to get dbt to work.