NVIDIA / modulus

Open-source deep-learning framework for building, training, and fine-tuning deep learning models using state-of-the-art Physics-ML methods
https://developer.nvidia.com/modulus
Apache License 2.0
797 stars 174 forks source link

🐛[BUG]: Licence check looks at files that are not in the git index #468

Open akshaysubr opened 2 months ago

akshaysubr commented 2 months ago

Version

main

On which installation method(s) does this occur?

Source

Describe the issue

Many times, there is a need to have some custom python scripts, markdown files, etc in the source tree to aid development. These are never in the git index and will always be untracked and linter failures on these files encourage the use of --no-verify when making commits and undo the benefits of pre-commit hooks. There is also no point in running linters and license checks on these files.

It would be good to only run linters on files that have been staged. There are existing tools that can help with this, eg:

Minimum reproducible example

touch my_untracked_script.py
make license

Relevant log output

❯ git status
On branch distributed-manager-group-query-refactor
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   Makefile
        modified:   modulus/distributed/__init__.py
        modified:   modulus/distributed/manager.py
        modified:   modulus/distributed/utils.py
        modified:   test/distributed/test_manager.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        Dockerfile.eos
        examples/generative/corrdiff/channels_last.py
        examples/generative/corrdiff/fused_group_norm.py
        test/edge_stats.json
        test/node_stats.json

❯ make license
pre-commit run license -a
license..................................................................Failed
- hook id: license
- exit code: 1

License check config:
{
    "copyright_file": "copyright.txt",
    "dir": "../../",
    "exclude-dir": [
        "../../modulus/internal",
        "../../docs/"
    ],
    "include-ext": [
        ".py",
        ".yaml",
        ".ci",
        ".release",
        "Dockerfile"
    ]
}
/home/akshays/Codes/Modulus/modulus-core/examples/generative/corrdiff/fused_group_norm.py has less header lines than the copyright template
/home/akshays/Codes/Modulus/modulus-core/examples/generative/corrdiff/channels_last.py has less header lines than the copyright template
test_header.py found the following files that might not have a copyright header:
/home/akshays/Codes/Modulus/modulus-core/examples/generative/corrdiff/fused_group_norm.py
/home/akshays/Codes/Modulus/modulus-core/examples/generative/corrdiff/channels_last.py
Traceback (most recent call last):
  File "/home/akshays/Codes/Modulus/modulus-core/test/ci_tests/header_check.py", line 210, in <module>
    main()
  File "/home/akshays/Codes/Modulus/modulus-core/test/ci_tests/header_check.py", line 203, in main
    assert len(problematic_files) == 0, "header test failed!"
AssertionError: header test failed!

make: *** [Makefile:31: license] Error 1

Environment details

No response