linters do not ignore `target/` directory after `make dev`

cfm commented 1 year ago

Description

After #6828, make dev now yields a target/ folder, which fails several linters under make lint. Like Bandit in #6843, they should ignore target/.

Steps to Reproduce

make dev
make lint

Expected Behavior

Linters pass.

Actual Behavior

Linters fail on generated code in the target/ tree.

legoktm commented 1 year ago

Which linters are failing for you?

user@dev ~/g/f/securedrop> USE_PODMAN=1 make dev
███ Starting development server...
STEP 1/19: FROM ubuntu@sha256:450e066588f42ebe1551f3b1a535034b6aa46cd936fe7f2c6b0d72997ec61dbd
STEP 2/19: ARG USER_NAME
--> Using cache b7805eaa383de86bc3bc04662dcbfc218a285842b198397d54e70703a1c02382
--> b7805eaa383d
STEP 3/19: ENV USER_NAME ${USER_NAME:-root}
--> Using cache a6eac2f0441d5d752c6586e42a62cf7014ae18e04c7757e686e21854b7616bec
--> a6eac2f0441d
STEP 4/19: ARG USER_ID
--> Using cache d8bce6cc088afca85316e4a785ae8720815d7ec1f426bb51e4ab057f4659ae0b
--> d8bce6cc088a
STEP 5/19: ENV USER_ID ${USER_ID:-0}
--> Using cache 19cd52cecb5fc37aceb38a473047466a2e6ffaf992a5cc7451490cc440a497ad
--> 19cd52cecb5f
STEP 6/19: RUN apt-get update && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install -y                        apache2-dev coreutils vim                        python3-pip python3-all python3-venv virtualenv python3-dev libssl-dev                        gnupg2 redis-server git curl wget                        enchant libffi-dev sqlite3 gettext sudo tor basez pkg-config
--> Using cache 8a05d613c124c0e252f8d60d572a77bc02226af4cb5bcdeec727a79fa2b65690
--> 8a05d613c124
STEP 7/19: ENV RUST_VERSION 1.69.0
--> Using cache 8ca7ce23af91716b50edfbb7a0a94d1e24099862aef5e68c6c335e6713597d3b
--> 8ca7ce23af91
STEP 8/19: ENV RUSTUP_VERSION 1.24.3
--> Using cache fcd544c159b4d3b275d7bc1404d29c713ae1e639086d8ded6b9f004d3412be8d
--> fcd544c159b4
STEP 9/19: ENV RUSTUP_INIT_SHA256 3dc5ef50861ee18657f9db2eeb7392f9c2a6c95c90ab41e45ab4ca71476b4338
--> Using cache 96a267f2d15183493777a6d43692297ef5077aabc2757054200e1a1272103987
--> 96a267f2d151
STEP 10/19: ENV RUSTUP_HOME /opt/rustup
--> Using cache 5dd74e681a9ba18f421b1f985a7a4e0d51895fbc5f4eebc9de23e239408a9a60
--> 5dd74e681a9b
STEP 11/19: ENV CARGO_HOME /opt/cargo
--> Using cache 5b14b182b0b00c10812a84415ecac30135a33361127ee514ab085e8c842beaa9
--> 5b14b182b0b0
STEP 12/19: RUN TMPDIR=`mktemp -d` && cd ${TMPDIR}         && curl --proto '=https' --tlsv1.2 -OO -sSf https://static.rust-lang.org/rustup/archive/${RUSTUP_VERSION}/x86_64-unknown-linux-gnu/rustup-init         && echo "${RUSTUP_INIT_SHA256} *rustup-init" | sha256sum -c -         && chmod +x rustup-init         && ./rustup-init --default-toolchain=${RUST_VERSION} --profile minimal -y         && cd && rm -rf ${TMPDIR}
--> Using cache 1334bafc6c054106a0905cc358fa7db8aebbc36d9c344d32ebfe1a0c2a115d4b
--> 1334bafc6c05
STEP 13/19: ENV CARGO_REGISTRIES_CRATES_IO_PROTOCOL sparse
--> Using cache 850988c669c5f8bf17423228178626930aa654572598868168f3e82c5899bbdc
--> 850988c669c5
STEP 14/19: COPY requirements requirements
--> Using cache b22e2ea68d865b2a1ab3cbfe54e6048160bc720be46dd6132c774f8c4166f7be
--> b22e2ea68d86
STEP 15/19: RUN python3 -m venv /opt/venvs/securedrop-build &&     /opt/venvs/securedrop-build/bin/pip3 install --no-deps --require-hashes -r requirements/python3/build-requirements.txt
--> Using cache 0246bea5ac3095a6cd0be6fbb3ca1ae7a2d4f273d01caddfce8a726c7ae52a37
--> 0246bea5ac30
STEP 16/19: RUN python3 -m venv /opt/venvs/securedrop-app-code &&     /opt/venvs/securedrop-app-code/bin/pip3 install --no-deps --require-hashes -r requirements/python3/bootstrap-requirements.txt &&     /opt/venvs/securedrop-app-code/bin/pip3 install --no-deps --require-hashes -r requirements/python3/test-requirements.txt &&     /opt/venvs/securedrop-app-code/bin/pip3 install --no-deps --require-hashes -r requirements/python3/requirements.txt
--> Using cache 457fa9cabda0f5d8c355eb6da5fb6bc9714c3d0cfad257373bf3bf9d5423bd2c
--> 457fa9cabda0
STEP 17/19: RUN if test $USER_NAME != root ; then useradd --no-create-home --home-dir /tmp --uid $USER_ID $USER_NAME && echo "$USER_NAME ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers ; fi &&     chown -R $USER_NAME.$USER_NAME /opt/venvs/
--> Using cache 66985a954e11284c88841d952010ecb714a699959630efabfbafeb5a48363bff
--> 66985a954e11
STEP 18/19: STOPSIGNAL SIGKILL
--> Using cache 850681fde8d38bff7316fe83258d05929a5b9add743a835c07cecc2a48d94fce
--> 850681fde8d3
STEP 19/19: EXPOSE 8080 8081 5909
--> Using cache 8ee0bc24988c2ef88be5cd7a02f20f7bcc938eeee815d8b73a485a4ca3007c34
COMMIT securedrop-slim-focal-py3
--> 8ee0bc24988c
Successfully tagged localhost/securedrop-slim-focal-py3:latest
8ee0bc24988c2ef88be5cd7a02f20f7bcc938eeee815d8b73a485a4ca3007c34
************************************************************
Exposed services will be available on localhost at
Source interface: http://127.0.0.1:8080
Journalist interface: http://127.0.0.1:8081
************************************************************
rm: cannot remove '/dev/random': Device or resource busy
ln: failed to create symbolic link '/dev/random': File exists
Unable to replace /dev/random
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at /opt/venvs/securedrop-app-code/bin/python3
📡 Using build options compatibility from pyproject.toml
    Finished dev [unoptimized + debuginfo] target(s) in 0.04s
📖 Found type stub file at redwood.pyi
📦 Built wheel for CPython 3.8 to /home/user/github/freedomofpress/securedrop/target/dev/wheels/redwood-0.1.0-cp38-cp38-linux_x86_64.whl
Processing /home/user/github/freedomofpress/securedrop/target/dev/wheels/redwood-0.1.0-cp38-cp38-linux_x86_64.whl
Installing collected packages: redwood
Successfully installed redwood-0.1.0
WARNING: You are using pip version 21.1.1; however, version 23.1.2 is available.
You should consider upgrading via the '/opt/venvs/securedrop-app-code/bin/python3 -m pip install --upgrade pip' command.
main: /var/lib/securedrop/db.sqlite
^Cmake: *** [Makefile:239: dev] Error 2
(2) user@dev ~/g/f/securedrop> make lint
███ Linting Ansible configuration...
/bin/bash: line 1: molecule: command not found
make: *** [Makefile:92: ansible-config-lint] Error 127
(2) user@dev ~/g/f/securedrop> source .venv/bin/activate.fish
(.venv) user@dev ~/g/f/securedrop> make lint
███ Linting Ansible configuration...
--> Test matrix

└── ansible-config
    └── verify

--> Inventory /home/user/github/freedomofpress/securedrop/molecule/ansible-config/../../install_files/ansible-base/group_vars linked to /home/user/.cache/molecule/securedrop/ansible-config/inventory/group_vars
--> Inventory /home/user/github/freedomofpress/securedrop/molecule/ansible-config/../../install_files/ansible-base/host_vars linked to /home/user/.cache/molecule/securedrop/ansible-config/inventory/host_vars
--> Scenario: 'ansible-config'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/user/github/freedomofpress/securedrop/molecule/ansible-config/tests/...
    ============================= test session starts ==============================
    platform linux -- Python 3.8.16, pytest-7.2.0, pluggy-0.13.1
    rootdir: /home/user/github/freedomofpress/securedrop/molecule, configfile: pytest.ini
    plugins: testinfra-5.3.1, xdist-3.0.2, forked-1.3.0
collected 27 items                                                             

    tests/test_play_configuration.py ...........................             [100%]

    ============================== 27 passed in 0.79s ==============================
Verifier completed successfully.

███ Linting application code...

███ Running black check...
All done! ✨ 🍰 ✨
222 files would be left unchanged.

███ Running isort check...
Skipped 11 files

███ Running flake8...

███ Linting application templates...

███ Linting shell scripts...
ShellCheck - shell script analysis tool
version: 0.9.0
license: GNU General Public License, version 3
website: https://www.shellcheck.net

███ Running mypy type checking...
./securedrop/bin/dev-shell: line 44: nc: command not found
🔗 Found pyo3 bindings
🐍 Found CPython 3.8 at /opt/venvs/securedrop-app-code/bin/python3
📡 Using build options compatibility from pyproject.toml
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
📖 Found type stub file at redwood.pyi
📦 Built wheel for CPython 3.8 to /home/user/github/freedomofpress/securedrop/target/dev/wheels/redwood-0.1.0-cp38-cp38-linux_x86_64.whl
Processing ./target/dev/wheels/redwood-0.1.0-cp38-cp38-linux_x86_64.whl
Installing collected packages: redwood
Successfully installed redwood-0.1.0
WARNING: You are using pip version 21.1.1; however, version 23.1.2 is available.
You should consider upgrading via the '/opt/venvs/securedrop-app-code/bin/python3 -m pip install --upgrade pip' command.
Success: no issues found in 170 source files

███ Linting YAML files...

cfm commented 1 year ago

Via both Docker and USE_PODMAN=1, including after a refreshing rm -rf .venv && make venv:

user@sd-dev:~/securedrop$ make dev
[...]
^C
(.venv) user@sd-dev:~/securedrop$ source .venv/bin/activate
(.venv) user@sd-dev:~/securedrop$ make lint
[...]
would reformat target/cargo-dev/registry/src/index.crates.io-6f17d22bba15001f/unicode-normalization-0.1.22/scripts/unicode.py

Oh no! 💥 💔 💥
3 files would be reformatted, 220 files would be left unchanged.
make: *** [Makefile:72: check-black] Error 1

legoktm commented 1 year ago

Ohh, black had already reformatted it so it didn't show up for me. Once I rm -rf target/ then it started failing again.

legoktm commented 1 year ago

Both black and isort have flags to automatically use rules defined in .gitignore, will submit a PR for that in a bit.

freedomofpress / securedrop