Establish test coverage plan and implement CICD

[ericerway]

This issue is for the following packages:

[ ] venia-concept [ ] pwa-buildpack [ ] peregrine [ ] pwa-module [ ] pwa-devdocs

Description

As a developer I want to ensure full test coverage for PWA Studio and the Venia PWA storefront so that I create a quality application. This coverage will help identify issues that can be submitted to Github for triage and assignment.

Assumptions

• CircleCI is currently used for pwa-studio
• MFTF where possible; this issue is identify how this works
• Considerations for on-prem and cloud deployment (as appropriate)

Tasks

• [ ] Review current approach with CircleCI w/r/t CICD
• [ ] Review and propose plan (and stories) to align with MFTF

[dmtrkad]

• [ ] Review the current approach with CircleCI w/r/t CICD (3 SP)
• [ ] Review and propose the plan (and stories) to align with MFTF
  1. Need details

Q&A

1. Do we need to add UI integration tests running using CI/CD ?
2. What kind of integration tests do we need to run on the build basis?
3. Do we need to cover venia concept within integration tests?

Acceptance criteria 1 R&D of current approaches

[ericerway]

• Need help from @zetlen here as well; from a product standpoint I want full coverage/integration

[zetlen]

CircleCI is good for unit tests and integration tests. We build our tests on PRs, so our usage of the Circle open-source plan is fairly high already. It seems sensible that a test suite would run on another platform.

End-to-end testing is expensive and time-consuming, though it's crucial to a finished product. Our QA friend at EPAM should probably research how MFTF might be integrated here; you can stand up the staging server and run a WebDriver suite against it, for instance, so I think MFTF ought to be fine.

End-to-end tests might include:

• Full installation of PWA Studio and launch of the web server, on:
  • Windows 10+
  • Ubuntu 12+
  • Mac OSX 12+

• Product configuration in Venia

• Full checkout through Venia

• Matrix of various devices, form factors, OSes and browsers

• Other full scenarios

[dmtrkad]

Blocked by Devops team. To follow suggestion from Denis's side we need to proceed working on it since the moment devops team shares their results.

[awilcoxa]

@sharkySharks how relevant is this ticket given your current work?

[sharkySharks]

I don't have a ticket tracking my cicd work, which will include mftf test hookups. This can be assigned to me and put in progress if that makes sense, or I can make a new one.

[sharkySharks]

Cicd will move from cicleci to internally supported jenkins environments. 1 environment will be for the pwa team specifically, this will run ci functions and report back to github. The second jenkins instance will support multiple instances of magento as an api backend to pwa. Current work is focusing on getting the magento backend setup.

[sharkySharks]

Since this is a research task, I am closing this as research is complete. #924 & #925 have been created to track action from this research.