dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
395 stars 221 forks source link

[Bug] flake8 code smells #990

Closed danphenderson closed 2 months ago

danphenderson commented 7 months ago

Is this a new bug in dbt-spark?

Current Behavior

When running pre-commit, flake8 identifies some issues. Namely:

(dbt-spark) dbt-spark > pre-commit run --all-files                        
check yaml...............................................................Passed
check json...........................................(no files to check)Skipped
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
check for case conflicts.................................................Passed
black....................................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

dbt/adapters/spark/column.py:59:42: E231 missing whitespace after ':'
dbt/adapters/spark/column.py:60:42: E231 missing whitespace after ':'
dbt/adapters/spark/column.py:61:42: E231 missing whitespace after ':'
dbt/adapters/spark/column.py:62:42: E231 missing whitespace after ':'
dbt/adapters/spark/impl.py:239:47: E713 test for membership should be 'not in'
dbt/adapters/spark/python_submissions.py:50:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:64:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:96:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:118:31: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:129:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:197:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:211:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:232:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:248:20: E231 missing whitespace after ':'
dbt/adapters/spark/python_submissions.py:291:64: E231 missing whitespace after ':'
tests/unit/utils.py:242:72: E201 whitespace after '{'
tests/unit/utils.py:242:78: E202 whitespace before '}'
tests/unit/utils.py:242:101: E201 whitespace after '{'
tests/unit/utils.py:242:110: E202 whitespace before '}'

mypy.....................................................................Passed

Also, the flake8 config says this:

exclude = test

I believe it was intended to ignore tests. But since the tests namespace has few errors, it would be better practice to fix the errors and remove the exclude statement from the .flake8 config.

Expected Behavior

pre-commit

Steps To Reproduce

Run pre-commit run --all-files

Additional Context

(dbt-spark) dbt-spark > pip freeze | grep flake8 
flake8==6.1.0
colin-rogers-dbt commented 2 months ago

This has been fixed with our recent updates to linting