search

ZupIT / horusec

Horusec is an open source tool that improves identification of vulnerabilities in your project with just one command.
https://horusec.io/
Apache License 2.0
1.14k stars 184 forks source link

tests: create unit tests for rules of horusec-engine #630

Open matheusalcantarazup opened 3 years ago

matheusalcantarazup commented 3 years ago

We currently only have unit tests for the Leaks rules. We need to implement tests for other languages/technologies. These tests need to validate both safe code and insecure code to ensure that false positive and false negative coverage.

Theses tests should be added on the respective packages for each language that engine support. These packages can be founded on engines package.

To implement these tests we can use the tests already created for Leaks as example.

These tests should fill an array of testutil.RuleTestCase with all scenarios of tests. In the TestRulesVulnerableCode test fill the name, rule, src and findings fields, and in the TestRulesSafeCode test the name, rule and src fields.

The name field should have the rule id as in the existing examples of Leaks. The findings field must contain a list of vulnerabilities that the rule informed in the rule parameter must return using as input the src field which contains the vulnerable code. For better organization it is recommended that vulnerable and safe codes be created in a file samples_test.go like in leaks tests and referenced in the test cases.

Tests to be implemented of the leaks language:
- [x] HS-LEAKS-1 - [x] HS-LEAKS-2 - [x] HS-LEAKS-3 - [x] HS-LEAKS-4 - [x] HS-LEAKS-5 - [x] HS-LEAKS-6 - [x] HS-LEAKS-7 - [x] HS-LEAKS-8 - [x] HS-LEAKS-9 - [x] HS-LEAKS-10 - [x] HS-LEAKS-11 - [x] HS-LEAKS-12 - [x] HS-LEAKS-13 - [x] HS-LEAKS-14 - [x] HS-LEAKS-15 - [x] HS-LEAKS-16 - [x] HS-LEAKS-17 - [x] HS-LEAKS-18 - [x] HS-LEAKS-19 - [x] HS-LEAKS-20 - [x] HS-LEAKS-21 - [x] HS-LEAKS-22 - [x] HS-LEAKS-23 - [x] HS-LEAKS-24 - [x] HS-LEAKS-25 - [x] HS-LEAKS-26 - [x] HS-LEAKS-27 - [x] HS-LEAKS-28
Tests to be implemented of the csharp language:
- [x] HS-CSHARP-1 - [x] HS-CSHARP-2 - [x] HS-CSHARP-3 - [x] HS-CSHARP-4 - [x] HS-CSHARP-5 - [x] HS-CSHARP-6 - [x] HS-CSHARP-7 - [x] HS-CSHARP-8 - [x] HS-CSHARP-9 - [x] HS-CSHARP-10 - [x] HS-CSHARP-11 - [x] HS-CSHARP-12 - [x] HS-CSHARP-13 - [x] HS-CSHARP-14 - [x] HS-CSHARP-15 - [x] HS-CSHARP-16 - [x] HS-CSHARP-17 - [x] HS-CSHARP-18 - [x] HS-CSHARP-19 - [x] HS-CSHARP-20 - [x] HS-CSHARP-21 - [x] HS-CSHARP-22 - [x] HS-CSHARP-23 - [x] HS-CSHARP-24 - [x] HS-CSHARP-25 - [x] HS-CSHARP-26 - [x] HS-CSHARP-27 - [x] HS-CSHARP-28 - [x] HS-CSHARP-29 - [x] HS-CSHARP-30 - [x] HS-CSHARP-31 - [x] HS-CSHARP-32 - [x] HS-CSHARP-33 - [x] HS-CSHARP-34 - [x] HS-CSHARP-35 - [x] HS-CSHARP-36 - [x] HS-CSHARP-37 - [x] HS-CSHARP-38 - [x] HS-CSHARP-39 - [x] HS-CSHARP-40 - [x] HS-CSHARP-41 - [x] HS-CSHARP-42 - [x] HS-CSHARP-43 - [x] HS-CSHARP-44 - [x] HS-CSHARP-45 - [x] HS-CSHARP-46 - [x] HS-CSHARP-47 - [x] HS-CSHARP-48 - [x] HS-CSHARP-49 - [x] HS-CSHARP-50 - [x] HS-CSHARP-51 - [x] HS-CSHARP-52 - [x] HS-CSHARP-53 - [x] HS-CSHARP-54 - [x] HS-CSHARP-55 - [x] HS-CSHARP-56 - [x] HS-CSHARP-57 - [x] HS-CSHARP-58 - [x] HS-CSHARP-59 - [x] HS-CSHARP-60 - [x] HS-CSHARP-61 - [x] HS-CSHARP-62 - [x] HS-CSHARP-63 - [x] HS-CSHARP-64 - [x] HS-CSHARP-65 - [x] HS-CSHARP-66 - [x] HS-CSHARP-67 - [x] HS-CSHARP-68 - [x] HS-CSHARP-69 - [x] HS-CSHARP-70 - [x] HS-CSHARP-71 - [x] HS-CSHARP-72 - [x] HS-CSHARP-73 - [x] HS-CSHARP-74
Tests to be implemented of the dart language:
- [x] HS-DART-1 - [x] HS-DART-2 - [x] HS-DART-3 - [x] HS-DART-4 - [x] HS-DART-5 - [x] HS-DART-6 - [x] HS-DART-7 - [x] HS-DART-8 - [x] HS-DART-9 - [x] HS-DART-10 - [x] HS-DART-11 - [x] HS-DART-12 - [x] HS-DART-13 - [x] HS-DART-14 - [x] HS-DART-15 - [x] HS-DART-16 - [x] HS-DART-17
Tests to be implemented of the java language:
- [x] HS-JAVA-1 - [x] HS-JAVA-2 - [x] HS-JAVA-3 - [x] HS-JAVA-4 - [x] HS-JAVA-5 - [x] HS-JAVA-6 - [x] HS-JAVA-7 - [x] HS-JAVA-8 - [x] HS-JAVA-9 - [x] HS-JAVA-10 - [x] HS-JAVA-11 - [x] HS-JAVA-12 - [x] HS-JAVA-13 - [x] HS-JAVA-14 - [x] HS-JAVA-15 - [x] HS-JAVA-16 - [x] HS-JAVA-17 - [x] HS-JAVA-18 - [x] HS-JAVA-19 - [x] HS-JAVA-20 - [x] HS-JAVA-21 - [x] HS-JAVA-22 - [x] HS-JAVA-23 - [x] HS-JAVA-24 - [x] HS-JAVA-25 - [x] HS-JAVA-26 - [x] HS-JAVA-27 - [x] HS-JAVA-28 - [ ] HS-JAVA-29 - [ ] HS-JAVA-30 - [ ] HS-JAVA-31 - [ ] HS-JAVA-32 - [ ] HS-JAVA-33 - [ ] HS-JAVA-34 - [ ] HS-JAVA-35 - [ ] HS-JAVA-36 - [ ] HS-JAVA-37 - [ ] HS-JAVA-38 - [ ] HS-JAVA-39 - [ ] HS-JAVA-40 - [ ] HS-JAVA-41 - [ ] HS-JAVA-42 - [ ] HS-JAVA-43 - [ ] HS-JAVA-44 - [ ] HS-JAVA-45 - [ ] HS-JAVA-46 - [ ] HS-JAVA-47 - [ ] HS-JAVA-48 - [ ] HS-JAVA-49 - [ ] HS-JAVA-50 - [ ] HS-JAVA-51 - [ ] HS-JAVA-52 - [ ] HS-JAVA-53 - [ ] HS-JAVA-54 - [ ] HS-JAVA-55 - [ ] HS-JAVA-56 - [ ] HS-JAVA-57 - [ ] HS-JAVA-58 - [ ] HS-JAVA-59 - [ ] HS-JAVA-60 - [ ] HS-JAVA-61 - [ ] HS-JAVA-62 - [ ] HS-JAVA-63 - [ ] HS-JAVA-64 - [ ] HS-JAVA-65 - [ ] HS-JAVA-66 - [ ] HS-JAVA-67 - [ ] HS-JAVA-68 - [ ] HS-JAVA-69 - [ ] HS-JAVA-70 - [ ] HS-JAVA-71 - [ ] HS-JAVA-72 - [ ] HS-JAVA-73 - [ ] HS-JAVA-74 - [ ] HS-JAVA-75 - [ ] HS-JAVA-76 - [ ] HS-JAVA-77 - [ ] HS-JAVA-78 - [ ] HS-JAVA-79 - [ ] HS-JAVA-80 - [ ] HS-JAVA-81 - [ ] HS-JAVA-82 - [ ] HS-JAVA-83 - [ ] HS-JAVA-84 - [ ] HS-JAVA-85 - [x] HS-JAVA-86 - [ ] HS-JAVA-87 - [ ] HS-JAVA-88 - [ ] HS-JAVA-89 - [ ] HS-JAVA-90 - [ ] HS-JAVA-91 - [ ] HS-JAVA-92 - [ ] HS-JAVA-93 - [ ] HS-JAVA-94 - [ ] HS-JAVA-95 - [ ] HS-JAVA-96 - [ ] HS-JAVA-97 - [ ] HS-JAVA-98 - [ ] HS-JAVA-99 - [ ] HS-JAVA-100 - [ ] HS-JAVA-101 - [ ] HS-JAVA-102 - [ ] HS-JAVA-103 - [ ] HS-JAVA-104 - [x] HS-JAVA-105 - [x] HS-JAVA-106 - [ ] HS-JAVA-107 - [ ] HS-JAVA-108 - [ ] HS-JAVA-109 - [ ] HS-JAVA-110 - [x] HS-JAVA-111 - [ ] HS-JAVA-112 - [ ] HS-JAVA-113 - [ ] HS-JAVA-114 - [ ] HS-JAVA-115 - [ ] HS-JAVA-116 - [ ] HS-JAVA-117 - [ ] HS-JAVA-118 - [ ] HS-JAVA-119 - [ ] HS-JAVA-120 - [ ] HS-JAVA-121 - [ ] HS-JAVA-122 - [ ] HS-JAVA-123 - [ ] HS-JAVA-124 - [ ] HS-JAVA-125 - [ ] HS-JAVA-126 - [ ] HS-JAVA-127 - [ ] HS-JAVA-128 - [ ] HS-JAVA-129 - [ ] HS-JAVA-130 - [ ] HS-JAVA-131 - [ ] HS-JAVA-132 - [ ] HS-JAVA-133 - [x] HS-JAVA-134 - [ ] HS-JAVA-135 - [ ] HS-JAVA-136 - [ ] HS-JAVA-137 - [ ] HS-JAVA-138 - [ ] HS-JAVA-139 - [ ] HS-JAVA-140 - [ ] HS-JAVA-141 - [ ] HS-JAVA-142 - [ ] HS-JAVA-143 - [x] HS-JAVA-144 - [x] HS-JAVA-145 - [x] HS-JAVA-146 - [x] HS-JAVA-147 - [x] HS-JAVA-148 - [x] HS-JAVA-149
Tests to be implemented of the java and kotlin languages for JVM based:
- [ ] HS-JVM-1 - [ ] HS-JVM-2 - [ ] HS-JVM-3 - [ ] HS-JVM-4 - [ ] HS-JVM-5 - [ ] HS-JVM-6 - [ ] HS-JVM-7 - [ ] HS-JVM-8 - [ ] HS-JVM-9 - [ ] HS-JVM-10 - [ ] HS-JVM-11 - [ ] HS-JVM-12 - [ ] HS-JVM-13 - [ ] HS-JVM-14 - [ ] HS-JVM-15 - [ ] HS-JVM-16 - [ ] HS-JVM-17 - [ ] HS-JVM-18 - [ ] HS-JVM-19 - [ ] HS-JVM-20 - [ ] HS-JVM-21 - [ ] HS-JVM-22 - [ ] HS-JVM-23 - [ ] HS-JVM-24 - [ ] HS-JVM-25 - [ ] HS-JVM-26 - [ ] HS-JVM-27 - [ ] HS-JVM-28 - [ ] HS-JVM-29 - [ ] HS-JVM-30 - [ ] HS-JVM-31 - [ ] HS-JVM-32 - [ ] HS-JVM-33 - [ ] HS-JVM-34 - [ ] HS-JVM-35 - [ ] HS-JVM-36 - [ ] HS-JVM-37 - [ ] HS-JVM-38 - [ ] HS-JVM-39 - [ ] HS-JVM-40
Tests to be implemented of the kubernetes files:
- [x] HS-KUBERNETES-1 - [x] HS-KUBERNETES-2 - [x] HS-KUBERNETES-3 - [x] HS-KUBERNETES-4 - [x] HS-KUBERNETES-5 - [x] HS-KUBERNETES-6 - [x] HS-KUBERNETES-7 - [x] HS-KUBERNETES-8 - [x] HS-KUBERNETES-9
Tests to be implemented of the nginx files:
- [x] HS-NGINX-1 - [x] HS-NGINX-2 - [x] HS-NGINX-3 - [x] HS-NGINX-4
Tests to be implemented of the javascript language:
- [x] HS-JAVASCRIPT-1 - [x] HS-JAVASCRIPT-2 - [x] HS-JAVASCRIPT-3 - [x] HS-JAVASCRIPT-4 - [x] HS-JAVASCRIPT-5 - [x] HS-JAVASCRIPT-6 - [x] HS-JAVASCRIPT-7 - [x] HS-JAVASCRIPT-8 - [x] HS-JAVASCRIPT-9 - [x] HS-JAVASCRIPT-10 - [x] HS-JAVASCRIPT-11 - [x] HS-JAVASCRIPT-12 - [x] HS-JAVASCRIPT-13 - [x] HS-JAVASCRIPT-14 - [x] HS-JAVASCRIPT-15 - [x] HS-JAVASCRIPT-16 - [x] HS-JAVASCRIPT-17 - [x] HS-JAVASCRIPT-18 - [x] HS-JAVASCRIPT-19 - [ ] HS-JAVASCRIPT-20 - [ ] HS-JAVASCRIPT-21 - [ ] HS-JAVASCRIPT-22 - [ ] HS-JAVASCRIPT-23 - [ ] HS-JAVASCRIPT-24 - [ ] HS-JAVASCRIPT-25 - [ ] HS-JAVASCRIPT-26 - [ ] HS-JAVASCRIPT-27 - [ ] HS-JAVASCRIPT-28 - [ ] HS-JAVASCRIPT-29 - [ ] HS-JAVASCRIPT-30 - [ ] HS-JAVASCRIPT-31 - [ ] HS-JAVASCRIPT-32 - [ ] HS-JAVASCRIPT-33 - [ ] HS-JAVASCRIPT-34 - [ ] HS-JAVASCRIPT-35 - [ ] HS-JAVASCRIPT-36 - [ ] HS-JAVASCRIPT-37 - [ ] HS-JAVASCRIPT-38 - [ ] HS-JAVASCRIPT-39 - [ ] HS-JAVASCRIPT-40 - [ ] HS-JAVASCRIPT-41 - [ ] HS-JAVASCRIPT-42 - [ ] HS-JAVASCRIPT-43 - [ ] HS-JAVASCRIPT-44 - [ ] HS-JAVASCRIPT-45 - [ ] HS-JAVASCRIPT-46 - [ ] HS-JAVASCRIPT-47 - [ ] HS-JAVASCRIPT-48 - [ ] HS-JAVASCRIPT-49 - [ ] HS-JAVASCRIPT-50 - [ ] HS-JAVASCRIPT-51 - [ ] HS-JAVASCRIPT-52 - [ ] HS-JAVASCRIPT-53
Tests to be implemented of the swift language:
- [x] HS-SWIFT-1 - [x] HS-SWIFT-2 - [x] HS-SWIFT-3 - [x] HS-SWIFT-4 - [x] HS-SWIFT-5 - [x] HS-SWIFT-6 - [x] HS-SWIFT-7 - [x] HS-SWIFT-8 - [x] HS-SWIFT-9 - [x] HS-SWIFT-10 - [x] HS-SWIFT-11 - [x] HS-SWIFT-12 - [x] HS-SWIFT-13 - [x] HS-SWIFT-14 - [x] HS-SWIFT-15 - [x] HS-SWIFT-16 - [x] HS-SWIFT-17 - [x] HS-SWIFT-18 - [x] HS-SWIFT-19 - [x] HS-SWIFT-20 - [x] HS-SWIFT-21 - [x] HS-SWIFT-22 - [x] HS-SWIFT-23 - [x] HS-SWIFT-24
filipemelo commented 3 years ago

Looking into rules_test.go, I think we could improve csharp rules, so we can analyze if the command injection and xpath injection isn't sanityzing the input, so we should not release a false-positive.

nathanmartinszup commented 3 years ago

Looking into rules_test.go, I think we could improve csharp rules, so we can analyze if the command injection and xpath injection isn't sanityzing the input, so we should not release a false-positive.

Hello @filipemelo.

Any improvements to the rules are very welcome!

Currently, we only have a regex engine, so somethings can be hard to check some things with it. It's already in our plans we have a semantic analysis to improve the engine's assertiveness.

Thank you for your interest in contributing and any questions we are at your disposal.

filipemelo commented 3 years ago

Looking into rules_test.go, I think we could improve csharp rules, so we can analyze if the command injection and xpath injection isn't sanityzing the input, so we should not release a false-positive.

Hello @filipemelo.

Any improvements to the rules are very welcome!

Currently, we only have a regex engine, so somethings can be hard to check some things with it. It's already in our plans we have a semantic analysis to improve the engine's assertiveness.

Thank you for your interest in contributing and any questions we are at your disposal.

1) How can I help in samantic analysis? Are any issue to this? 2) Can I write code for all csharp rules or should I let some easy tasks for hacktoberfest event?

nathanmartinszup commented 3 years ago

Looking into rules_test.go, I think we could improve csharp rules, so we can analyze if the command injection and xpath injection isn't sanityzing the input, so we should not release a false-positive.

Hello @filipemelo. Any improvements to the rules are very welcome! Currently, we only have a regex engine, so somethings can be hard to check some things with it. It's already in our plans we have a semantic analysis to improve the engine's assertiveness. Thank you for your interest in contributing and any questions we are at your disposal.

  1. How can I help in samantic analysis? Are any issue to this?
  2. Can I write code for all csharp rules or should I let some easy tasks for hacktoberfest event?

Hi again @filipemelo.

  1. We do not have any open issue, we are at a very early stage and studying the best way to do it in our scenario. In case you want to know more about what we did until now, you can check this branch and also this commit which contains a explanation of what we are thinking.
  2. About writing all csharp rules, it will be much more than welcome. We still have a lot of other rules to test, so you don't have to worry about that.

Finally, thank you again for your contribution, it is very important to us.

filipemelo commented 3 years ago

About HS-CSHARP-3 - SCS0007 - XML eXternal Entity Injection (XXE).#SCS007 This only happens to version prior to 4.5.2 .Net Framework, wich is each time less used. (release 4.5.2 - 2014-05-05) Should we implement this and release false positive for beyond this version?

nathanmartinszup commented 3 years ago

About HS-CSHARP-3 - SCS0007 - XML eXternal Entity Injection (XXE).#SCS007 This only happens to version prior to 4.5.2 .Net Framework, wich is each time less used. (release 4.5.2 - 2014-05-05) Should we implement this and release false positive for beyond this version?

@filipemelo.

As we still don't have any way to verify the version, I believe it would be worth reducing the confidence and add the vulnerable versions in the description.