Create a functional test for content preview

[sebastienros]

cf https://github.com/OrchardCMS/OrchardCore/issues/15181

[Piedone]

Eh? https://github.com/OrchardCMS/OrchardCore/pull/11194

[hishamco]

I planned for a while to introduce Playwright as a testing framework to handle various types of testing, if all agree I will prepare a demo

[sebastienros]

It's fine using playwright, but the simplicity of the current functional tests should be kept. Noting that as simple as it is, nobody creates new ones :/ Maybe at some point we'll start asking for some in PRs. I have been personally shying about it, but on my other repositories it's usually well accepted, though these are more low level libraries.

[hishamco]

I like software testing very much, that's why I like other repos: YesSQL, Fluid, and Parlot :) I was planned to create an OS testing library specialized for OC, something similar to what Lombiq did but simpler :)

[Piedone]

UI testing is not simple and can't be done in a simple way either. You have to have complexity if you want to actually get some value out of the tests.

Under https://github.com/OrchardCMS/OrchardCore/pull/11194 I've implemented this. I really fail to see the value in reinventing the wheel with Playwright or Cypress.

Believe me, it starts small, like "I just need this test to click around a bit", but it doesn't until you have intricate knowledge of browser behavior, the UI testing ecosystem, CI systems, .NET idiosyncracies, the logging infrastructure, and an immense amount of other stuff. UI testing is not unit testing where everything just works.

[sebastienros]

Not reinventing the wheel. Cypress has been there for years in OC already, working fine, we just need to add scenarios with features. What I am asking here is just a request to the page to ensure the main element is rendered to check the content preview feature is not completely broken.

Example:

• Create a file like this
• Add a couple of Visit/Check to ensure the feature works

Idea is to find a balance between full coverage and not blocking external contributions because we ask too much.

[Piedone]

Yep, that's how it starts ;).

[hishamco]

UI testing is not unit testing where everything just work

Playwrite is promising there are many many features for UI & integration tests

[Skrypt]

Ok, we have also an issue with the current Cypress tests. The environment is set to Production and we run these tests on http instead of https and we would need to fix the npm package that @jptissot did years ago so that it start on https. That could be an easy fix if we had the npm package repository credentials but I'd rather invest time on Playwright at this point.

And that's me after wasting an hour figuring out the issue with my PR. :smile:

[hishamco]

but I'd rather invest time on Playwright at this point.

Let me know if you need a help on this, coz I already invested time on Playwright, it's promising

[Skrypt]

What I'm thinking is that we could easily keep Cypress old tests and create new ones with Playwright and then migrate Cypress ones to Playwright progressively.

[Skrypt]

@hishamco Please share a repository if you can. I've barely just started trying to use it with the VS Code extension. Question I'm asking myself is if we need to create a lot of boilerplate TS code like we have for Cypress (js npm package) to create tenants and else.

[Piedone]

Why bring in another JS testing framework, and with TS on top of it, when we can write tests in C#? Even with Playwright.

[Skrypt]

I need to see how it all works still, I've seen C# support but the VS Code extension seems to leverage the TS tests. I personally don't dislike TypeScript to be honest. It's easier to read/maintain than plain javascript at least. Now, not everything should/can be C# and Blazor. All of these have their own limitations/drawback. Need to live with it.

[hishamco]

Why bring in another JS testing framework, and with TS on top of it, when we can write tests in C#? Even with Playwright.

Totally agree, during my testing and investigation I'm using .NET. Probably I will do a demo if I have a time

[Skrypt]

I like the VS Code UI functionaly and the Chromium browser automated browsing display. Also the fact that we can record these tests and it will write code for it. Looks more like Cypress to me without the complexity. Playwright allows for both styles of tests client and server side so I would not limit myself to only one type of test.

[hishamco]

Hopefully, I will demo some of the Playwright features this week or after