google / pytype

A static type analyzer for Python code
https://google.github.io/pytype
Other
4.71k stars 274 forks source link

Add junit xml output format #363

Open UfoRobot opened 5 years ago

UfoRobot commented 5 years ago

This could allow to easily integrate pytype checks in many continuous integration environments.

I found a similar issue ( https://github.com/google/pytype/issues/92) but it was resolved with CSV formatting

sbs2001 commented 4 years ago

How is the junit xml suppsed to look like ?

The closest thing I could find was mypy's junit output eg :

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="mypy" skips="0" tests="1" time="0.937">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_8-linux" time="0.937">
    <failure message="mypy produced messages">
vulnerabilities/tests/conftest.py:26: error: Skipping analyzing 'pytest': found module but no type hints or library stubs
vulnerabilities/tests/test_import_cmd.py:25: error: Skipping analyzing 'django.core.management': found module but no type hints or library stubs
</failure>
  </testcase>
</testsuite>

So inferring from that, I guess pytype's junit ouput would roughly look like

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="pytype" skips="0" tests="1" time="0.937">
  <testcase classname="pytype" file="pytype" line="1" name="pytype-py3_8-linux" time="0.937">
    <failure message="pytype produced messages">
File "/home/shivam/coding/opensource/vulnerablecode/vulnerabilities/import_runner.py", line 152, in _process_added_advisories: Function _collect_package_urls was called with the wrong arguments [wrong-arg-types]
         Expected: (batch: Set[vulnerabilities.data_source.Advisory])
  Actually passed: (batch: vulnerabilities.data_source.Advisory)
</failure>
  </testcase>
</testsuite>

for the stdout error produced

File "/home/shivam/coding/opensource/vulnerablecode/vulnerabilities/import_runner.py", line 152, in _process_added_advisories: Function _collect_package_urls was called with the wrong arguments [wrong-arg-types]
         Expected: (batch: Set[vulnerabilities.data_source.Advisory])
  Actually passed: (batch: vulnerabilities.data_source.Advisory)

Basically wrapping the error's found by pytype in a <failure> tag .

Is this the expected junit output ? I haven't used junit so pardon me if this comes off silly ;)

adam-azarchs commented 3 years ago

More or less, yes. There's attributes in the schema which would be useful to fill in, such as severity levels and whatnot.

The main use case I'd have for this would be integration with jenkins dashboards.