search

ubiquity / ubiquity-dollar

Ubiquity Dollar (UUSD) smart contracts and user interface.
https://uad.ubq.fi
Apache License 2.0
34 stars 91 forks source link

Make echidna working #291

Closed 0xcodercrane closed 9 months ago

0xcodercrane commented 2 years ago

Problem

Echidna ci/cd script looks great but what we want here is to get the script working for even 1 ~ 2 solidity files. So we want to make the echidna script working for both BondingV2.sol and BondingShareV2.sol

Acceptance Criteria

zgorizzo69 commented 1 year ago

I am very interested in that issue. I am wondering from what branch to start ? I can't find the echidna ci/cd script in the development branch so I am guessing it is the fix/echidna branch right ? The thing is that branch has not been rebased on development so the first thing to do is a huge rebase IMO.

0xcodercrane commented 1 year ago

I will appreciate it if you complete this bounty. hmm, I prefer to take echidna script from fix/echidna and apply its changes to new branch forked from development. You don't need to start on fix/echidna branch. I made that branch to not miss the echidna script.

zgorizzo69 commented 1 year ago

Roger that we only miss a bounty for this issue

0x4007 commented 1 year ago

Echidna ci/cd script looks great but what we want here is to get the script working for even 1 ~ 2 solidity files. So we want to make the echidna script working for both BondingV2.sol and BondingShareV2.sol

Doesn't it make sense for the bounty to set up Echidna for every smart contract? I would increase the Profit label to 50% if so. Kind of wish we had Profit: <25% lol cause current price seems low for assessing Bonding/Staking.

ghost commented 1 year ago

I'm interested in this bounty How can I get started?

0x4007 commented 1 year ago

Some caveats:

Assuming both are ok with you, then this may be a good first bounty to attempt.

zgorizzo69 commented 1 year ago

Some caveats:

* zgo thinks that some prerequisite work might need to be finished before.

I am very interested in that issue. I am wondering from what branch to start ? I can't find the echidna ci/cd script in the development branch so I am guessing it is the fix/echidna branch right ? The thing is that branch has not been rebased on development so the first thing to do is a huge rebase IMO.

* Are you familiar with Echidna?

Assuming both are ok with you, then this may be a good first bounty to attempt.

Also you first need to identify what are the invariants that you want to preserve for BondingV2 and BondingShareV2. If invariants don't make sense echidna testing is useless

rndquu commented 1 year ago

@0xcodercrane @zgorizzo69

Perhaps we can modify this task to "create foundry fuzzing tests for all of the ubiquity contracts"? As far as I understand foundry fuzzing tests do the same job as echidna.

0xcodercrane commented 1 year ago

Yep! exactly. echidna is originally fuzz tests in which the foundry has that feature embedded. I agree

0x4007 commented 1 year ago

I recall having this conversation some months ago. The conclusion was that Echidna handles a slightly different problem space. This should be confirmed before scrapping it.

gitcoindev commented 1 year ago

Hi! I have came across your project and would enjoy working on this.

A few words on my relevant background:

I used libfuzzer in the past for "traditional" fuzzing. It was mostly integration and results analysis. I tried slither for static analysis about 2-3 years ago.

Therefore, I do not have much Echidna experience yet but would like to check how useful it is.

My plan for this:

I will experiment today and tomorrow and give you a first update by Wednesday. I believe first thing will be to compile the contract with Echidna and @openzeppelin, might require some remappings.

After this task is completed and Echidna founds to be useful I would like to work on Echdina standard properties test https://github.com/crytic/properties/blob/main/PROPERTIES.md#erc20 (https://github.com/ubiquity/ubiquity-dollar/issues/587)

Resources:

If I will not be able to open a PR by Friday EOD I will step down from this task.

ubiquibot[bot] commented 1 year ago

Available commands

- /assign: Assign the origin sender to the issue automatically.
- /unassign: Unassign the origin sender from the issue automatically.
- /help: List all available commands.
- /payout: Disable automatic payment for the issue.
- /multiplier: Set bounty multiplier (for treasury)
- /allow: Set access control. (Admin Only)
- /wallet: <WALLET_ADDRESS | ENS_NAME>: Register the hunter's wallet address. 
  ex1: /wallet 0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266 
  ex2: /wallet vitalik.eth

@gitcoindev

gitcoindev commented 1 year ago

/start

ubiquibot[bot] commented 1 year ago

Skipping /start since no time labels are set to calculate the timeline

gitcoindev commented 1 year ago

/wallet 0x7e92476D69Ff1377a8b45176b1829C4A5566653a

ubiquibot[bot] commented 1 year ago

Updated the wallet address for @gitcoindev successfully! Your new address: 0x7e92476D69Ff1377a8b45176b1829C4A5566653a

gitcoindev commented 1 year ago

<1 week

gitcoindev commented 1 year ago

Hi @pavlovcik I noticed the @ubiquibot time is set for <1 day, would be grateful if you could adjust this to <1 week.

rndquu commented 1 year ago

@gitcoindev sorry but this task is not ready yet, we're using foundry which has a built-in fuzzing feature, pls wait while we update the description

gitcoindev commented 1 year ago

Hi @rndquu all right! Thank you for the heads up. I'd be grateful if you pinged me once the description is ready.

0x4007 commented 1 year ago

@gitcoindev sorry but this task is not ready yet, we're using foundry which has a built-in fuzzing feature, pls wait while we update the description

I thought that they cover different scenarios and so it makes sense to use both?

rndquu commented 1 year ago

@gitcoindev sorry but this task is not ready yet, we're using foundry which has a built-in fuzzing feature, pls wait while we update the description

I thought that they cover different scenarios and so it makes sense to use both?

Both echidna and foundry can be used for invariant testing

The hardest part is to put those invariants in the right places because this requires an overall project understanding at a decent level

As long as we use foundry as a dev framework we'd better stick to a tool that is already implemented in our environment

molecula451 commented 9 months ago

seems a duplicate of #587 where there are discussions to decide if echidna will be part of the repo

ubiquibot[bot] commented 9 months ago 
# Issue was not closed as completed. Skipping.
0x4007 commented 9 months ago

seems a duplicate of #587 where there are discussions to decide if echidna will be part of the repo

My understanding is that Echidna covers different problems compared to the foundry fuzzer so we should use it.

molecula451 commented 9 months ago

yeah but let's gather another perspective in a different issue

rndquu commented 9 months ago

seems a duplicate of #587 where there are discussions to decide if echidna will be part of the repo

My understanding is that Echidna covers different problems compared to the foundry fuzzer so we should use it.

Foundry supports both fuzzing tests and invariant (i.e. echidna) tests

0x4007 commented 9 months ago

ChatGPT response:

I did a quick search for more information and here's what I found.

Echidna is a prominent tool for fuzzing Ethereum smart contracts. It's developed in Haskell and focuses on property-based testing. Echidna uses sophisticated grammar-based fuzzing campaigns based on a contract's ABI to test user-defined predicates or Solidity assertions. It's particularly known for its unique approach to 'property-based fuzzing', where it tries to falsify user-defined invariants instead of looking for crashes like a traditional fuzzer. This capability allows it to write custom analyses for more complex contracts. Echidna's focus is on Ethereum Virtual Machine (EVM) bytecode, using tools like cryptic compile for testing contracts【9†source】【10†source】【11†source】.

Foundry, on the other hand, is a newer toolkit that includes fuzzing capabilities along with other features for Ethereum development. Its design emphasizes integration with the overall development workflow, making it easy to use alongside other Foundry tools. Foundry is known for its efficiency and the speed of its operations, which can be a significant advantage in certain development scenarios.

Comparing the two, it's clear that while there may be some overlap in core functionality, each tool has unique features and strengths. Echidna's property-based fuzzing is highly specialized and suited for complex contract testing, while Foundry's integrated approach and speed make it attractive for a more streamlined development process. The choice between these tools would largely depend on the specific needs and preferences of the development team, as well as the complexity of the contracts being tested.

Therefore, Foundry fuzzing doesn't necessarily render Echidna fuzzing redundant. Instead, they can be viewed as complementary tools, each excelling in different aspects of smart contract testing. In some cases, using both might offer the most comprehensive testing coverage for smart contracts.

https://chat.openai.com/c/84accfbb-7198-419d-bfde-246c719770c1

molecula451 commented 9 months ago

echidna scope highly different from foundry built in fuzzing, since pavlovcik wants it will best to just have it rendering but instead executing this on a newer issue, with new specific inputs for the task, upcoming

0x4007 commented 9 months ago

Well I want to make it very clear that I have no experience using Echidna and almost none with Foundry. I haven't even heard a lot of people on Twitter talking about Echidna lately. Perhaps its obsolete? I'm not sure.

molecula451 commented 9 months ago

Yes echidna came first before foundry, and it was one of the first fuzzing tool for ethereum (there are papers that analyze echidna) it was mainly targeted to fuzzing ERC20 in initials stages, will have to research but it only supports a set of properties to fuzz

gitcoindev commented 9 months ago

Interestingly, I found a yet another player in the game called Medusa which apparently won the comparison:

https://github.com/devdacian/solidity-fuzzing-comparison

0x4007 commented 9 months ago

the goal is finding the best performance "out of the box" with as little guidance & tweaking as possible.

Source: https://github.com/devdacian/solidity-fuzzing-comparison/blob/main/README.md

I think thats pretty nifty. Although I am more keen to adopt technologies that I see the people I follow start talking about. However if its easy to set up then I see no issue with adding this to the CI.