search

onflow / developer-grants

Grants for developers that contribute to the broader developer ecosystem
Apache License 2.0
50 stars 18 forks source link

Cadence Testing Framework Major Uplift #216

Closed m-Peter closed 12 months ago

m-Peter commented 1 year ago

Cadence Testing Framework

Grant category

Please select one:

Description

The Cadence Testing Framework has come a long way, after the completion of https://github.com/onflow/developer-grants/issues/148. It has been successfully used as the testing suite of https://github.com/onflow/hybrid-custody, enhancing the project's CI pipeline and DevEx. A number of other projects have been using the framework as well, including the FungibleToken and NonFungibleToken standards plus some other community projects.

Problem statement

After these first two iterations, we have identified some missing features which will make the framework feature-complete. Up until now, there have been two types of testing: unit & integration tests. These two required a very different approach of testing, which resulted in some inconsistencies and a lack of features for both testing types. Unit tests were very limited in testing very primitive contracts, and integration tests could not make use of deployed contract types, which is one of the most needed features. This lack of features is described here: https://github.com/onflow/cadence-tools/issues/209

Proposed solution

To overcome all these limitations, we will unify the backend environment for both unit & integration tests. The emulator blockchain will no longer be exposed to the testing framework, as it will be just a singleton. Test code will have to deploy a contract prior to using it. The accounts where contracts are to be deployed, will be specified in the familiar flow.json config file, in a newly-introduced network (testing). We have borrowed an idea from the Flow Playground, and will bootstrap the framework with 10 predefined accounts, for contract deployment. More details can be found in this post: https://forum.onflow.org/t/major-uplift-for-cadence-testing-framework/5232

Impact

Provide test / development parity, by allowing developers to write unit/integration tests in Cadence (https://12factor.net/dev-prod-parity). Developers will also be able to measure how well their tests cover their contracts, and gain sufficient confidence before deploying to testnet, and more importantly on mainnet.

Milestones and funding

Milestone Deliverables Timeline Risks USD proposal
1 - Implementation Merge https://github.com/onflow/cadence-tools/pull/210 & https://github.com/onflow/flow-cli/pull/1227 4 weeks - 5000
2 - Adoption Update 5 projects to use the new testing API 2 weeks - 2500

Total funding proposed: 7500 $

Team

Name Role Bio Contact
Ardit Marku Back-end Engineer A seasoned back-end engineer with a focus on problem solving and designing robust software systems. Over 9 years of experience building high-traffic search engines, marketplaces, exchanges and in-house analytics solutions. Led cross-functional teams to deliver both web and mobile apps, participating in all steps of the software development lifecycle. Open source lover and advocate. markoupetr@gmail.com m_peter
m-Peter commented 1 year ago

An example of unit testing the built-in Crypto contract, in the cadence repository: https://github.com/onflow/cadence/pull/2856

m-Peter commented 1 year ago

Status Update:

1st Milestone:

2nd Milestone:

As soon as a new flow-cli release is made available, I can proceed with the above 5 PRs which migrate the testing suite to the new API.

cc @franklywatson

m-Peter commented 1 year ago

Besides the Flow core repositories, I am adding one more migration from a community-driven project:

m-Peter commented 1 year ago

Hey @franklywatson The 2nd milestone is also completed. The testing suite for 5 Flow core projects has been migrated. The relevant PRs are all merged. I have migrated another community project :pray: I believe we can mark the entire grant as completed :tada:

franklywatson commented 12 months ago

Great work. Thanks!