jmcameron
commented
1 year ago I've done some initial experiments with Playwright and think that will work for our end-to-end regtests. I've converted 6 regtests and I'm in the process of converting others. There will be some difficulties:
- When I started, I had some difficulties with operations on locators producing "Target Closed" errors. So I was using deprecated functions (eg, deprecated
page.click(selector)instead of non-deprecatedpage.locator(selector).click()). I was able to get the non-deprecated calls to work. I thought there was some issue with Playwright handling our older version of AngularJS. However, later I was able to get the regular non-deprecated functions working so I think we are okay on that issue (not sure what I did differently). - There are many end-to-end tests and related shared functions for interacting with the website. I was not able to make all the shared functions to work exactly like the old functions, so will be necessary to modify all regtests as they are converted to the new framework (Playwright). This means there is a lot of work ahead of us to migrate the end-to-end tests.
There is good news about using Playwright:
- Playwright seems to be popular and is being actively maintained and developed
- It is pretty fast (so far it is faster than one test per second - on average)
- It does not depend on webdriver/chromedriver.
- It works with Firefox too (although I'm currently only trying it with Chrome).
- Even though the tests are done headless, playwright has the ability to trace and replay operations and let you see (visually) what is happening in each step of the regtests.
Because Protractor is now obsolete for end-to-end tests, we need to convert the end-to-end tests to some other framework. See Issue https://github.com/IMA-WorldHealth/bhima/issues/6669 for details. Based on the feedback in that issue and some experimentation, I think we should try to convert the end-to-end tests to Playwright.
Trying the experimental version of the End-to-End tests with Playwright
- To try the current playwright end-to-end tests, look at my branch: [https://github.com/jmcameron/bhima/tree/playwright-test](https://github.com/jmcameron/bhima/tree/playwright-testl). - Update ('yarn') - You also need to install playwright via npm: - `npm install playwright` - Verify the two versions match: - grep '@playwright/test@' yarn.lock - npx playwright --version - Use the command: `yarn test:e2e` Eslint update - This PR includes an optional update to support eslint-ing jsdoc blocks. We encourage you to add JSDoc blocks above all your function definitions in javascript files. - To enable it on your local installs and via VS Code, do this: - cd client; ln -s ../.eslintrc.local.json .eslintrc.json; cd .. - cd server; ln -s ../.eslintrc.local.json .eslintrc.json; cd .. - cd test; ln -s ../.eslintrc.local.json .eslintrc.json; cd .. - The symbolic links created here are ignored by git so that commits will not save them. Therefore the using the ESlint extensions for JSDoc will only occur in your local setup, not on the master in github. - The JSDOc eslint extensions are in the file: .eslintrc.local.json - Note that these JSDoc extensions only apply to javascript files under the three directories (client, server, and test). Therefore they do not apply to top-level files, files within the `sh` or `utiliities` directories. - This work-around is because hound does not work well with the JSDoc extensions for eslint.Log of updates (open to see list)
- 2023-08-23 Playwright end-to-end tests working in CI/semaphore - 2023-08-17 Got most tests working - Split new playwright end-to-end tests to stock tests and non-stock tests - 'npx playwright install` - `npm run test:e2e` - `npm run test:e2e-stock` - Non-stock tests: 463 passed (26 skipped, none failed, 13.0min, 1 workers - NOT parallel) - Stock tests: 81 passed (2 skipped, none failed, 2.6min, 1 workers - NOT parallel) - 2023-08-03 Completed migration - Completed migrating stock tests and remaining components - Completed 100% of 144 migration tasks - 546 passed (16 skipped, 10 failed, 15.0min, 1 workers - NOT parallel) - 2023-07-21 PM Migrated journal tests - Completed 135 of 144 (including 98% of test directories, 64/65) - 466 passed (15 skipped, 8 failed, 12.3min, 1 workers - NOT parallel) - 2023-07-21 Migrated reports tests - Migrated 'report' tests (11 subdirectories, 23 files) - Note: Most stock exit reports skipped due to loading problem - Migrated components: bhCronSelect, bhEntityGroupSelect, bhMultipleCashBoxSelect, bhPeriodSelection, bhReportPeriodSelect, bhYesNoRadio - Completed 134 of 144 (including 97% of test directories, 63/65) - 454 passed (15 skipped, 8 failed, 11.8min, 1 workers - NOT parallel) - 2023-07-19 Migrated more tests - Migrated tests: costCenter, user (6 files) - Migrated components: bhAllocationBasisSelect (new), bhFiscalPeriodSelect, bhPercentageInput, bhServicesMultipleSelect - Note: Although the costCenter tests were migrated, most are skipped due to a problem during center creation - Completed 126 of 144 (including 94% of test directories, 61/65) - 417 passed (12 skipped, 3 failed, 10.4min, 1 workers - NOT parallel) - 2023-07-18 Migrated more tests - Migrated tests: visits, vouchers, ward (with 3 subdirectories) - Migrated components: bhCashBoxSelect, bhRoomSelect, bhWardSelect - Completed 118 of 142 (including 87% of test directories, 56/64) - 399 passed (5 skipped, 5 failed, 10.7min, 1 workers - NOT parallel) - 2023-07-14 Migrated more tests - Migrated tests: staffingIndice, subsidies, suppliers, surveyForm, transactionType, verificationLinks - Migrated components: bhDataCollector, bhSurveyFormTypeSelect, bhSurveyListSelect - Completed 112 of 142 (including 83% of test directories, 53/64) - 361 passed (3 skipped, 5 failed, 8.7min, 1 workers - NOT parallel) - 2023-07-13 Migrated more tests - Migrated purchases, rubrics, rubricsConfig, services tests - Migrated bhAddItem, bhProjectSelect, bhPurchaseStatusSelect components - Created new bhCostCenterSelect component - Completed 103 of 142 (including 72% of test directories, 46/64) - 327 passed (3 skipped, 2 failed, 2.5min, 8 workers) - Note: I ran the test with 1 worker (not in parallel). All tests pass but requires 8 minutes. - 2023-07-12 Migrated more tests - Migrated payrollConfig, payrollProcess, pinceList tests - Migrated components: bhAccountConfigSelect, bhEmployeeConfigSelect, bhIprConfigSelect, bhPayrollPeriodSelect, bhPayrollStatusSelect, bhRubricConfigSelect, and bhWeekCofnigSelect - Completed 94 of 141 (including 64% of test directories, 41/64) - Fixed a couple of problems when running all tests in parallel - 299 passed (2 skipped, 1 failed, 2.0min, 8 workers) - 2023-07-10 Migrated more test - Migrated iprTaxes, iprTaxesConfig, tags, weekendConfig and component bhIprScale - Completed 84 of 141 (including 61%, 39/64 test directories) - 282 passed (4 skipped, 4 failed, 2.0min, 8 workers) - 2023-07-07 Migrate more tests - migrated navigation, invoicingFees tests - Hand problems on one test in invoicingFees and it is currently skipped - Implemented GridAction utility - Completed 78 of 141 (including 53% of test directories) - 255 passed (2 skipped, 6 failed, 1.9min, 8 workers) - 2023-07-05 Migrate more tests - Migrated indicators_files and inventory tests - Migrated 5 components - Modified some tests not to fail when run in full parallel mode - Completed 75 of 141 (including 50% of test directories) - 252 passed (1 skipped, 4 failed, 1.9min, 8 workers) - 2023-07-03 Updates - update dependencies - Modified some tests not to fail when run in full parallel mode - Completed 68 of 141 (including 47% of test directories) - 235 passed (1 skipped, 1.7min, approx 2.3 secs/test, 8 workers) - 2023-06-30 Migrated more tests - Migrated holidays and component bhEmployeeSelect - Completed 68 of 141 (including 47% of test directories) - 233 passed (1 skipped, 2 failed, 1.9min) - Failed test are due to interference between related tests in different directories. These tests all work when the directory containing them is run by itself. - 2023-06-29 Migrated more tests - Migrated fiscalYears, functions, grades and component bhSurveyFormSelect - Completed 66 of 141 (including 44% of test directories) - 227 passed (1 skipped, 3 failed, 1.9min) - 2023-06-28 Migrated more tests and components - Migrated employees tests and bhFonctionSelect + bhGradeSelect components - Completed 61 of 141 (including 39% of test directories) - 208 passed (1 skipped, 4 failed, 1.8min) - Most failures were in the patients and employee tests. These tests need to be refactored in order to avoid conflicts when the tests are executed in different orders. Fixed a few tests by allowing several different counts. - 2023-06-14 Migrated more tests - Migrated f_employee_config tests and bhCheckTree.js component - Completed 58 of 141 (including 39% of test directories) - 200 passed (1 skipped) in 1.4 minutes (approx 2.4 secs/test, 8 workers, 2nd try) - 2023-06-13 Migrated more tests - Migrated tests for dataCollectionManagement and edit_journal - Completed 56 of 141 (including 34% of test directories) - 193 passed (1 skipped) in 1.4 minutes (approx 2.3 secs/test, 8 workers, 2nd try) - 2023-06-08 Migrated another test - Migrated tests for creditor_groups - Completed 54 of 141 (including 36% of test directories) - 182 passed (1 skipped) in 1.3 minutes (approx 2.3 secs/test, 8 workers) - 2023-06-07 Migrated more tests - Migrated tests for choiceListManagement, configurationAnalysisTools - Migrated components bhAccountReferenceSelect, beAnalysisToolTypeSelect - Completed 53 of 141 (including 33% of test directories) - 178 passed (1 skipped) in 1.2 minutes (approx 2.5 secs/test, 8 workers) - Tests are still a bit flakey. Getting all the tests to pass required 3 tries. - 2023-06-06 Migrated more tests - Migrated tests for exchange, errors - Completed 49 of 141 (including 30% of test directories) - 168 passed (1 skipped) in 1.2 minutes (approx 2.3 secs/test, 8 workers) - 2023-06-05 Minor updates - Minor updates to keep up with dependencies in master - Changed the default time that NotifyService success/failure messages get shown from 3 seconds to 5 seconds to reduce the number timeout failures that the Playwright end-to-end tests are experiencing. It seems to help but has not eliminated all random failures - Completed 47 of 141 (including 29% of test directories) - 164 passed (1 skipped) in 1.2 minutes (approx 2.3 secs/test, 8 workers) - 2023-06-02 Various updates - Minor updates pushed to PR on May 30, 31, and June 1 - Updated to master and pushed a change to the PR supporting using JSDoc with eslint locally (see previous section) - 2023-04-21 Finished patient tests - Pushed changes to PR that finished the 'patient' tests (10 files total) - Impelmented 3 more components (ChoiceListSelect, DiagnosisSelect, PatientGroupSelect) - Completed 46 of 141 (including 29% of test directories) - 164 passed (1 skipped) in 48.6 seconds (2.1 secs/test, 8 workers) - 2023-04-20 Keep up with master - Pushed changes to the PR to keep up with bump updates to the master - Continue working on patient tests (not pushed to PR yet) - 2023-04-19 Worked on new tests (patient) - Nothing released yet - 2023-04-18 Submitted a PR enable the already-completed Playwright end-to-end tests - Primarily for testing and to see how it works for others - When I've fixed to some of the repeatability problems, we get it working with github / bors - 2023-04-14 Implemented cashbox and debtors tests - Completed 41 of 141 (including 27% of test directories) - One old test is breaking occasionally, skipping it until I can fix it - 121 passed (2 skipped) in 48.6 seconds (2.5 secs/test, 8 workers) - 2023-04-13 Implemented more tests - Implemented account-config and accounts tests - Completed 39 of 141 (deleted sub-listings of some of tests) - 110 passed (1 skipped) in 41.9 seconds (2.6 secs/test, 8 workers) - 2023-04-12 Implemented more tests - Got accountReferenceType tests and the rest of account-references tests working - Got TestUtils.uiSelect() working for 'exact' and 'fullWord' option. - Completed 44 of 148 migration tasks - 93 passed (1 skipped) in 34.4 seconds (2.7 secs/test, 8 workers) - 2023-04-11 Got most of account-references test working - These tests were difficult to debug due to (probable) bugs in Playwright when executing javacript in the client when clicking on buttons. - Completed 42 of 148 migration tasks - 87 passed (1 skipped) in 34.6 seconds (2.5 secs/test, 8 workers) - 2023-04-10 Fixed an issue with TU.locator() chaining - Fixed a problem with TU.locator() that was preventing chaining with it. Went back and fixed all prior tests. - Completed 42 of 148 migration tasks - 80 passed (1 skipped) in 30.5 seconds (2.6 secs/test, 8 workers) - 2023-04-07 Working on account-reference tests - Implemented the bhAccountReferenceType component - Completed 42 of 148 migration tasks - 80 passed (1 skipped) in 30.7 seconds (2.6 secs/test, 8 workers) - 2023-04-06 Continuing to work on 'cash' tests - Fixed timeout issue with cash.spec.ts. Migrated cash/registry.spec.js - Implemented a couple more components - Completed 48 of 148 migration tasks - 77 passed (1 skipped) in 27.9 seconds (2.76 secs/test, 8 workers) - 2023-04-05 Continuing working on 'cash' tests - Got cash.spec.js tests working when run alone. But some of them are failing when run with the full test suite. I need to debug this problem. - Migrated 11 more components - Completed 36 of 148 tasks - Currently 69 tests - 2023-04-04 Continuing working on 'cash' tests - Updated branch with recent updates to master - Migrated bhCurrencyInput and bhCurrencySelect components - Completed 24 of 146 conversion tasks - 2023-04-03 Started on 'cash' tests - Updated branch with recent updates to master - 2023-03-31 Got account_statement tests working - Implemented most of the remaining functions in GridUtils - 59 passed (1 skipped) in 24.3 seconds (2.43 secs/test) - 2023-03-30 Added depots tests - Added depots.spec.js (and depots.page.js) including a couple of tests that were skipped in the past - Implemented bhDepotSelect - 55 passed (1 skipped) in 23.3 seconds (2.36 secs/test) - 2023-03-29 Worked to speed up tests - Figured out how to log in only once in each test file. This speeds things up from approximately 1 test per second to nearly 3 tests per second (for 45 tests). - Updated prot2play script to migrate files better (including changes reducing the number of logins, etc) - 2023-03-28 Worked on the last 'locations' test - Completed 19 of 146 (45 individual tests running) - The tests seem to be getting a bit flaky -- Need to track that down. It may be that we are creating/editing locations in two different test files. May have to make sure the names are unique and be more specific in selectors to avoid spill-over between tests files. - 2023-03-27 Started script to do partial automatic conversion to playwright (/sh/prot2play). - Competed 17 of 146 tests - 2023-03-24 - Added checklist for converting shared utilities - Started writing up conversion guide (in this comment, see below) - Completed 14 of 141 conversion tasks - 2023-03-23 Added progress check list of end-to-end tests in comments below - 2023-03-08 I have started converting our end-to-end tests to Playwright.Progress converting end-to-end tests to Playwright (64, open to see list)
- [x] account-reference (3 files) - [x] accountConfig (2 files, renamed account-config) - [x] accountReferenceType (2 files, renamed account-reference-type) - [x] account_statement (2 files, renamed account-statement) - [x] accounts (2 files) - [x] ~allocationCostCenters/distribution_feescenters* (2 files)~ - [x] allocationCostCenters/distribution_key* (2 files) - [x] ~breakEvenReference (2 files)~ - [x] cash (2 files) - [x] cashboxes (1 file) - [x] choiceListManagement (2 files) - [x] configurationAnalysisTools (2 files) - [x] creditor_groups (1 file) - [x] dataCollectorManagement (2 files) - [x] debtors (1 file) - [x] depots (2 files) - [x] displayMetadata (3 files) - [x] edit_journal (1 file) - [x] employees (6 files) - [x] enterprises (1 file) - [x] entities (2 files) - [x] errors (1 file) - [x] exchange (1 file) - [x] f_employees_config (2 files) - [x] feeCenter (2 files) - [x] fillForm (2 files) - [x] fiscalYears (1 file) - [x] functions (2 files) - [x] grades (2 files) - [x] holidays (2 files) - [x] indicators_files (2 files) - [x] inventory (2 files) - [x] invoicingFees (1 file) - [x] iprTaxes (2 files) - [x] iprTaxesConfig (2 files) - [x] journal (8 files) - [x] locations (5 files) - [x] login (1 file) - [x] navigation (1 file) - [x] offdays (2 files) - [x] patient (10 files) - [x] payrollConfig (2 files) - [x] payrollProcess (3 files) - [x] price_list (3 files) - [x] purchases (4 files) - [x] ~receipt_modal (1 file)~ - [x] reports (11 subdirectories, 23 files) - [x] rubrics (2 files) - [x] rubricsConfig (2 files) - [x] services (2 files) - [x] settings (1 file) - [x] staffingIndice (2 files) - [x] stock (23 files) - [x] subsidies (1 file) - [x] suppliers (1 file) - [x] surveyForm (2 files) - [x] tags (2 files) - [x] transactionType (1 file) - [x] user (6 files) - [x] verificationLinks (1 file) - [x] visits (2 files) - [x] vouchers (5 files) - [x] ward (3 directories) - [x] weekendConfig (2 files)Progress converting end-to-end shared utilities to Playwright (open to see list)
- [x] FormUtils.js --> TestUtils.js - [x] GridAction.js - [x] GridRow.js - [x] GridUtils.js - [x] helpers.js - [x] search.page.jsProgress converting end-to-end components to Playwright (74, open to see list)
- [x] bhAccountConfigSelect - [x] bhAccountReferenceSelect - [x] bhAccountReferenceTypeSelect - [x] bhAccountSelect - [x] bhAddItem - [x] bhAllocationBasisSelect (new) - [x] bhAnalysisToolTypeSelect - [x] bhCashBoxSelect - [x] bhCheckboxTree - [x] bhChoiceListSelect - [x] bhCostCenterSelect - [x] bhCronSelect - [x] bhCurrencyInput - [x] bhCurrencySelect - [x] bhDataCollector - [x] bhDateEditor - [x] bhDateInterval - [x] bhDatePicker - [x] bhDebtorGroupSelect - [x] bhDepotSelect - [x] bhDiagnosisSelect - [x] bhEmployeeConfigSelect - [x] bhEmployeeSelect - [x] bhEntityGroupSelect - [x] bhEntitySelect - [x] bhEntityTypeSelect - [x] bhFilters - [x] bhFindInvoice - [x] bhFindPatient - [x] bhFiscalPeriodSelect - [x] bhFiscalYearPeriodSelect - [x] bhFiscalYearSelect - [x] bhFluxSelect - [x] bhFonctionSelect - [x] bhGenderSelect - [x] bhGradeSelect - [x] bhInputText - [x] bhInventoryGroupSelect - [x] bhInventorySelect - [x] bhInventoryTypeSelect - [x] bhIprConfigSelect - [x] bhIprScale - [x] bhLocationSelect - [x] bhModalAction - [x] bhMultipleCashBoxSelect - [x] bhMultipleDepotSearchSelect - [x] bhPatientGroupSelect - [x] bhPayrollPeriodSelect - [x] bhPayrollStatusSelect - [x] bhPercentageInput - [x] bhPeriodSelect - [x] bhPeriodSelection - [x] bhProjectSelect - [x] bhPurchaseStatusSelect - [x] bhReportPeriodSelect - [x] bhReportSource - [x] bhRequisitionSelect - [x] bhRoomSelect - [x] bhRubricConfigSelect - [x] bhServiceOrDepot - [x] bhServiceSelect - [x] bhServicesMultipleSelect - [x] bhStockEntryExitType - [x] bhSupplierSelect - [x] bhSurveyFormSelect - [x] bhSurveyFormTypeSelect - [x] bhSurveyListSelect - [x] bhTagSelect - [x] bhTransactionTypeSelect - [x] bhUserSelect - [x] bhWardSelect - [x] bhWeekConfigSelect - [x] bhYesNoRadios - [x] notifyProtractor => Playwright Conversion Guide
- See [Playwright conversion guide](https://playwright.dev/docs/protractor) for details on converting `element()`, `by.css()`, etc. - At the top of the test file, insert `const { test, expect } = require('@playwright/test');` - The tests run in English. Therefore, it may be necessary to change some French labels and text used to find elements to English counterparts. - Conversions | Protractor | Playwright | | | --- | --- | --- | | `FU = require('../shared/FormUtils')` | `TU = require('../shared/TestUtils');` | | | `const { expect } = require('chai');` | `const { test, expect } = require('@playwright/test');` | 1 | | `describe(...)` | `test.describe(...)` | | | `before(() => helpers.navigate(path));` | `test.beforeEach(async ({ page }) => {...});` | 2 | | `it('description', async () =>Closes https://github.com/IMA-WorldHealth/bhima/issues/6669