ApeWorX / ape

The smart contract development tool for Pythonistas, Data Scientists, and Security Professionals
https://apeworx.io
Apache License 2.0
883 stars 132 forks source link

Add Slither scanning and Echidna fuzzing #2279

Closed YeagerAI-Bulat closed 1 month ago

YeagerAI-Bulat commented 1 month ago

Overview

I would like to see the integration of Slither and Echidna into the Ape framework to enhance the security testing and auditing of smart contracts.


Specification

  1. Slither Integration

    • Syntax: A new command such as ape test --slither should be introduced, which runs Slither on the target Solidity contracts within the Ape project.
    • Semantics:
      • Slither performs static analysis and outputs a detailed report, including possible vulnerabilities, code optimizations, and bug detection.
      • The output should be displayed within Ape’s CLI, allowing developers to quickly identify and address issues without switching environments.
      • Optionally, configuration files (e.g. slither.config.json, for enabling specific checks or formats) could be provided.
  2. Echidna Integration

    • Syntax: A command like ape test --echidna should be implemented, which runs Echidna fuzz testing on the project's smart contracts.
    • Semantics:
      • Echidna performs property-based testing by fuzzing contract inputs to verify correctness.
      • It should be possible to define properties in contract files (as per Echidna's property syntax) and run these tests seamlessly from the Ape CLI.
      • The test results (including any failing inputs) should be displayed in the Ape output.
      • Allow for passing configurations such as fuzzing depth, gas limits, or time limits via command-line flags.
  3. General Syntax

    • These commands can be run individually or in combination with Ape's existing testing commands (e.g., ape test --fuzz to combine Echidna with existing tests).
    • Both tools should be configurable through Ape's configuration files (e.g., ape-config.yaml), allowing developers to set default options for Slither and Echidna runs.

Dependencies

The implementation would likely depend on creating wrappers or plugins that bridge Ape with Slither and Echidna.

linear[bot] commented 1 month ago

APE-1814 Add Slither scanning and Echidna fuzzing

fubuloubu commented 1 month ago

Static analysis tools like slither can be run completely independently of ape, and it's highly recommended to run slither separately since it may have conflicting dependencies with ape (slither is also a Python project)

Echidna is an excellent fuzzing engine, and you can take full advantage of it alongside any ape project. To use it, you do have to design test harnesses in solidity, and ape can ignore those test files if you use the extension .t.sol

There is a fuzzing engine for python called Hypothesis that has actually been the inspiration for many other fuzzing engines, and that is the best match for ape since you can use it inside your ape tests (which are written in Python). There is another issue to add better support for Hypothesis to ape, however there is a tradeoff since ape implements full transaction validation and chain emulation where it can be very slow in practice. More purpose-built fuzzers like Echidna will be much faster because they do not do full transaction validation or attempt to emulate the chain in any way.

Hopefully that helps you understand the landscape a bit better!

fubuloubu commented 1 month ago

Closing this as duplicates of #363 #351 #1590