Fixes #11: Implements pysa tests in Github CI

[abishekvashok]

Adds python script which compares the issues pysa detects in delibrately_vulnerable_app and raise exceptions (return an error) when there is a mismatch. The output is also written to results.actual and printed.

Adds a github action called pysa that install the latest nightly version of pyre-check, runs setup.sh, and the run_integration_tests.py python script. It thereby, effectively validating the Pysa models.

Adds results.expected that will contain issues known to be true in the app. For now, it's just issues from rce.py, but can be expanded later on as more vulnerabilities are added to the flask app. The issues contained in it is in a concise form containing only the attributes- path, define, and code.

The python script, run_integration_tests.py reads issues in resulsts.expected, runs pysa, compares issues generated by pysa against those specified manually in results.expected and prints a list of issues which were detected in addition to the specified ones and/or the issues that weren't detected but were in resulsts.expected, in addition to raising a custom exception: PysaIssuesMismatchException.

Adds .gitignore containing list of directories generated by running pysa via the command pyre analyse --no-verify.

Modifies setup.sh to tailor to the github actions by creating a .pyre_configuration file with tain_models_path pointing to ../../stubs

Adds requirements.txt to contain requirements for the flask app. Currently it contains only flask but will be expanded later when more vulnerabilities are added to flask app.

Signed-off-by: Abishek V Ashok abishekvashok@gmail.com

[abishekvashok]

The Github Action can be found here: https://github.com/abishekvashok/pyre-check/runs/2887709253

It fails correctly as the nightly version of pysa can't detect the 3 errors in rce.py that the stable version can. (Tested it on my local machine as well, but unfortunately the nightly build can't detect the 3 issues)

[abishekvashok]

Finally, managed to get around pysa not showing issues by making it use models in the repository. Made those changes and excluded the python site-packages folder.

Now it detects all 3 issues, see this Github Action: https://github.com/abishekvashok/pyre-check/runs/2896936782?check_suite_focus=true

Also remember, current facebook/pyre-check repository contains models to generate 2 more issues, but we are currently running against our repository which is way behind. We may need to update the head of the repository and/or add those issues to true_issues.json, when these commits reach there (or sometime else, idk)

Thanks for @r0rshark for staying with me in this!

[abishekvashok]

Feel free to update this repo with the upstream one. It may also be easier if you open the pull request directly on the facebook/pyre-check repository so it has more visibility and other can comment

I will make a pull request there, I guess! I updated the repository nonetheless.

[abishekvashok]

Bang! Made the requested changes and squashed everything to a single commit. See the github action now: https://github.com/abishekvashok/pyre-check/actions/runs/973723685

[abishekvashok]

So as per suggestion I opned a PR to facebook/pyre-check https://github.com/facebook/pyre-check/pull/438 Closing this one there of.