search

notaryproject / roadmap

Roadmap for Notary Project
7 stars 6 forks source link

E2E Automation Testing for CLI commands #47

Closed iamsamirzon closed 1 year ago

iamsamirzon commented 2 years ago

Summary Automated Test Cases to be executed before every release Intended Outcome A test suite is created over time and added itertively over the lifetime of this project development Additional context Will update based on finalized design/spec in prior version of alpha release

FeynmanZhou commented 2 years ago

Hi @iamsamirzon ,

I assume this issue is requesting E2E testing for Notation CLI. We investigated the Golang-native E2E test framework in ORAS CLI and finally chose ginkgo+gomega.

We can reuse the same E2E test framework and solution in Notation CLI. How about transferring this issue to the Notation repo?

cc @yizha1 @dtzar

iamsamirzon commented 2 years ago

@FeynmanZhou - Yes, this is for the end to end testing of Notation CLI workflows. I suggest you leave this issue as is here, in the roadmap repo, but covert it into a "User Story" format. And then link it to the new issue/PR you create in "notation" to actually implement it. As I think, this issue here will become an epic with multiple end to end test cases implemented inside "notation" repo that will complete this "roadmap" item

cc: @vaninrao10

dtzar commented 2 years ago

We would have the user story here, but the actual work items in notation repo.

dtzar commented 1 year ago

Per the community call today, it would be good to have minimal E2E test coverage for RC.1 - such as for sign and verify workflow - and expand on this after.

JeyJeyGao commented 1 year ago

I will work on this task.

yizha1 commented 1 year ago

The E2E testing work is on hold for two reasons:

@dtzar @toddysm @vaninrao10 @shizhMSFT

yizha1 commented 1 year ago

updated to rc.2 per discussion.

yizha1 commented 1 year ago

Close as released in rc.2