search

schubmat / Custom-MADE

Custom-MAnangement of DEcision -- A DSL based approach to capture and manage your decisions during a software development project
MIT License
5 stars 3 forks source link

Feature/integration tests #4

Closed pcprinz closed 3 years ago

pcprinz commented 3 years ago

All unit tests and the integration tests pass within 188 seconds. To run the test firefox is mandatory. instructions in ./integration-tests/README.md

schubmat commented 3 years ago

@pcprinz You comitted the geckodriver as well. Is this one platform specific?

schubmat commented 3 years ago

@pcprinz Review in progress.

pcprinz commented 3 years ago

@pcprinz You comitted the geckodriver as well. Is this one platform specific?

Yes. this is the FireFox driver for linux, though the system only runs on linux. It's possible to refactor the tests to detect if they are started on linux / windows / macos and which browser is set to the default browser of the system, but this will lead to a huger programming task. Maybe for another featrue

schubmat commented 3 years ago

Just executed the test suite. For the bash script it's necessary to run:

inside the test suite folder.

Apart from that, the following three test cases did not pass. Looking at the failure reports it looks like the execution time might influence the result. When repeating them manually, I couldn't see the reason why they did not work. :man_shrugging:

=========== Test Report ===========

FAIL tests/integration.test.ts (54.55 s) ● Integration tests › IT-1

Failed: "Error on SignInPage.login(): Page was expected to be http://localhost:3000/home but is http://localhost:3000/"

  83 |    *   eine neue Datei hoch.
  84 |    */
> 85 |   test('IT-1', async () => {
     |   ^
  86 |     // login to accessable project
  87 |     const signIn = new SignInPage(driver);
  88 |     await signIn.navigate();

  at Env.it (node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:95:24)
  at Suite.<anonymous> (tests/integration.test.ts:85:3)
  at Object.<anonymous> (tests/integration.test.ts:23:1)

● Integration tests › IT-3

Failed: "Error on Project.validateProject('Project B'): Error on Projects.findProject('Project B'): No Project found with this name!"

  135 |    *   Rechner vorhanden sein.
  136 |    */
> 137 |   test('IT-3', async () => {
      |   ^
  138 |     // sign in and open accessable project
  139 |     const signIn = new SignInPage(driver);
  140 |     await signIn.navigate();

  at Env.it (node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:95:24)
  at Suite.<anonymous> (tests/integration.test.ts:137:3)
  at Object.<anonymous> (tests/integration.test.ts:23:1)

● Integration tests › IT-4

Failed: "Error on Project.exportProject('Project B'): Error on Projects.findProject('Project B'): No Project found with this name!"

  177 |    * - Es müssen im Anschluss zwei Zip-Dateien auf dem Rechner vorhanden sein.
  178 |    */
> 179 |   test('IT-4', async () => {
      |   ^
  180 |     // sign in and open accessable project
  181 |     const signIn = new SignInPage(driver);
  182 |     await signIn.navigate();

  at Env.it (node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:95:24)
  at Suite.<anonymous> (tests/integration.test.ts:179:3)
  at Object.<anonymous> (tests/integration.test.ts:23:1)
schubmat commented 3 years ago

@pcprinz Worked perfectly fine. ... However, I added the refactoring I talked about, which was not only a refactoring actually. I changed some things, like removing all alerts. Thus, I started fixing some of the test code, but I did not finish. My progress is not that fast and I am currently out of time. @pcprinz May be you can have a look at the remaining issues. Just a few notes:

pcprinz commented 3 years ago

1. closing the creation popup

The popup (that remains present after deleting a file) possibly overlays elements of the list which will lead to an "... cant scroll to element..." error. For that reason the popup is closed by pressing the add button again

2. No Project found with this name

I have to look this up. This can just happen when the list of projects is not yet loaded, but the las commit should have fixed that.

3. Delete > Find > no file found with this name

Can probably happen if:

4. Expect Assertions

The Promises of the Pages' functions should work with void results (for consistency reasons). In Order to test if the upload of a file was NOT successful the function must reject with an Error. This error will be used as message when a test expects to be successful, but fails.

If you want a test to expect that the upload fails (because of the wrong file extension) then you have to check that the function rejects an error (and what kind of message the error has). To do that we place the expect() in the catch() clause of the function (or a try-catch).

But when the function unexpectedly doesn't fail, then the expect() in the catch() clause will never be called. For that reason Jest provides the expect.assertions() function in order to catch up missed fails.

In your case teh function which was expected to fail, doesn't fail anymore (because the alert that shows "wrong extension" doesn't show up anymore). How do you want to test misbehaviour without alerts?

5. Separately run tests

the common way to run a test separately is to chain the test() or describe() function with only:

only() works just fine for every describe() and test()within a file, but not for multiple files somehow. skip() always works, but then you have to write it everywhere.

A temporary workaraound is to skip the describe() functions in the files you want to skip and then write only() for a particular function in the remaining file you want to test.

Edit: I'm already working on a faster solution in the next assignment