Open Keith-CY opened 6 months ago
WhiteMinds
commented
6 months ago In terms of automated testing, it is worth considering the introduction of some RPA (Robotic Process Automation) technologies, such as Alibaba Cloud RPA, UiBot, UiPath, and other similar products.
We can synchronize projects related to automation processes to a specific repository.
"按键精灵" lacks active maintenance and is not recommended for use.
Keith-CY
commented
6 months ago In terms of automated testing, it is worth considering the introduction of some RPA (Robotic Process Automation) technologies, such as Alibaba Cloud RPA, UiBot, UiPath, and other similar products.
We can synchronize projects related to automation processes to a specific repository.
"按键精灵" lacks active maintenance and is not recommended for use.
Quite informative about RPA, we should use it to facilitate interactive testing
FrederLu
commented
6 months ago There is a recording function in UiPath and OpenRpa, which will be more convenient during the editing steps and reduce learning time to a certain extent.
https://www.uipath.com/rpa/robotic-process-automation https://github.com/open-rpa/openrpa
silySuper
commented
6 months ago I searched a way to automate application UI in MacOs(which is free ,but need write code): Atomac+Accessibility Inspector(come with xcode)+Python
The way to open the app is as follows（notice the right sidebar）：
we need to download xcode and use Accessibility Inspector to locate element and operate specific function.
WhiteMinds
commented
6 months ago In the field of Robotic Process Automation (RPA), there aren't many software solutions that offer good support for multiple systems simultaneously. This means that if you want to test multiple systems (such as macOS + Windows), you would need to write two sets of testing code. Otherwise, testing only a single platform may not be comprehensive enough.
If you only need to test a single system (such as macOS), then this approach is fine.
However, if we need to support automation testing on multiple platforms, I suggest considering trying the Electron adapter of Playwright: https://playwright.dev/docs/api/class-electron
Keith-CY
commented
6 months ago In the field of Robotic Process Automation (RPA), there aren't many software solutions that offer good support for multiple systems simultaneously. This means that if you want to test multiple systems (such as macOS + Windows), you would need to write two sets of testing code. Otherwise, testing only a single platform may not be comprehensive enough.
If you only need to test a single system (such as macOS), then this approach is fine.
However, if we need to support automation testing on multiple platforms, I suggest considering trying the Electron adapter of Playwright: playwright.dev/docs/api/class-electron
Does it work for interactive testing? The tests of Neuron are categorized into 4 groups
Playwright is going to be used in automated UI/UX tests, but I'm not sure if it helps in interactive tests.
WhiteMinds
commented
6 months ago Does it work for interactive testing? The tests of Neuron are categorized into 4 groups
- automated tests of full node synchronization (with a dev node as the fixture)
- automated tests of light client synchronization (with a dev node as the fixture)
- automated UI/UX tests(with a preset database of Neuron as the fixture)
- interactive tests just before the release (with testnet node)
Playwright is going to be used in
automated UI/UX tests, but I'm not sure if it helps in interactive tests.
I haven't done any actual research, but based on my speculation, interactive testing should be its strong suit, and UX testing should include the interactive aspect.
Keith-CY
commented
6 months ago Does it work for interactive testing? The tests of Neuron are categorized into 4 groups
- automated tests of full node synchronization (with a dev node as the fixture)
- automated tests of light client synchronization (with a dev node as the fixture)
- automated UI/UX tests(with a preset database of Neuron as the fixture)
- interactive tests just before the release (with testnet node)
Playwright is going to be used in
automated UI/UX tests, but I'm not sure if it helps in interactive tests.I haven't done any actual research, but based on my speculation, interactive testing should be its strong suit, and UX testing should include the interactive aspect.
Yes, UX testing includes the interactive aspect. Or to say, interactive tests are complement to automated UX tests.
UI/UX tests were split into 2 automated and interactive because activities on mainnet/testnet are unpredictable. When those unpredictable parts are eliminated, e.g. being tested on devnet, it can be automated.
But before the release, neuron should be tested on testnet, so unpredicted activities are inevitable, and interactive tests would be added to handle them, e.g. transaction is committed much later than expected because the testnet is busy. If this case is automatic, it would hit timeout exception.
silySuper
commented
6 months ago I learned that Playwrite is used for software runned on Web browser. Does it can be used for app ?
WhiteMinds
commented
6 months ago I learned that Playwrite is used for software runned on Web browser. Does it can be used for app ?
Playwright provides experimental support for running on Electron: https://playwright.dev/docs/api/class-electron
The Neuron client is based on Electron.
WhiteMinds
commented
6 months ago There is also a recently popular library for desktop automation called nut.js : https://nutjs.dev/
Using this library, you can achieve cross-platform compatibility. It might be worth considering researching and exploring this library for development purposes.
Recently, Neuron discovered a critical bug in an upcoming release, and according to the developer's analysis, it has been present in previous versions. This raises the subtopic of the accuracy of test results. As we work on fixing this bug and move into the acceptance phase, I've noticed that the reporting time is significantly long. The primary reason for this delay is the time consumed in synchronization during each acceptance, highlighting the second subtopic: insufficient testing efficiency.
Improving the accuracy and efficiency of testing results is a common challenge, often addressed with the introduction of automation. However, before Neuron is released, it inevitably undergoes a manual full regression. Therefore, the focus narrows down to improving the accuracy and efficiency of the interactive testing process.
My suggestion is to introduce semi-automation to assist in this process.
Taking this bug as an example, the most noticeable symptom is the misidentification of transaction amounts in Nervos DAO. This error is easily detectable since Nervos DAO records shouldn't have such significant amount changes. Yet, in repetitive mechanical tasks, humans are prone to overlooking details, especially in familiar content.
For this example, I propose exporting the transaction list before and after testing and programmatically comparing them to ensure consistency. This involves a semi-automated process where manual data export is followed by automated verification through a program.
While the manual operation in this case is relatively simple, our test cases may involve more complex actions.
For instance, validating a multi-output transaction requires repetitive inputs of addresses and amounts, password input, and observation of balance and transaction record changes after a certain time.
Or, to verify that the sliding bar for the Nervos DAO deposit amount updates correctly in different positions, multiple repetitions of sliding actions are needed, with observation of the displayed amount after each operation.
Both examples can be accomplished using tools to perform the actions and ultimately rely on human judgment to determine if the results match expectations.
Especially in the first example, using a "按键精灵" to perform steps like "export transaction records A," "input addresses and amounts," "input password," "export transaction records B," and "remove transaction records A from B" allows for judgment based solely on the output of the last step. Furthermore, saving this result as a benchmark allows for automated checking of correctness in subsequent repetitions.
The key difference between this approach and full automation is that the various automated steps are not necessarily seamlessly connected but are triggered manually. This provides flexibility and a closer resemblance to the production environment compared to fully automated tests that can only run within test data.
In short, we should introduce some tools to automate interactive testing for accuracy and efficiency. It would be kinda close to fully automated testing but more flexible and real as the final step of full-regression test.